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FOREWORD 


The  research  presented  in  this  report  was  conducted  by  the  Infantry  Forces  Research  Unit 
(IFRU)  of  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  imder 
its  TRAINMOD:  Training  Modernization  (#204)  work  package.  TRAINMOD  involves 
addressing  training  issues  associated  vvith  new  Infantry  systems.  New  Infantry  systems,  such  as 
the  Land  Warrior  (LW),  have  computer  software  that  soldiers  and  leaders  must  use  to  execute 
their  mission.  It  is  anticipated  that  much  future  training  on  these  and  similar  “digital  systems” 
will  be  done  with  some  form  of  interactive  multi-media  or  computer-based  training  (CBT). 

Under  TRAINMOD,  a  series  of  experiments  was  begun  on  how  to  develop  CBT  to  train  the 
target  population  on  these  systems.  This  report  describes  the  results  from  the  first  experiment.  It 
examined  the  impact  of  the  amount  of  information  presented  in  a  lesson  as  well  as  the 
effectiveness  of  an  exploratory  mode  of  learning. 

The  results  provide  insights  into  the  design  of  effective  and  efficient  CBT  for  digital 
systems.  A  training  technique  that  divided  instructional  content  into  relatively  small  chucks  and 
then  presented  training  exercises  was  more  effective  than  a  technique  that  presented  large 
volumes  of  information  before  soldiers  had  training  exercises.  In  addition,  an  exploratory  mode 
of  learning  showed  promise  as  it  required  the  least  time,  but  it  did  not  yield  the  highest 
performance.  Variations  in  exploratory  modes  of  learning  that  incorporate  more  traditional 
training  techniques  may  be  more  effective.  The  different  training  techniques  were  applied  to  two 
different  domains:  learning  “facts”  and  learning  “software  interface  procedures.”  Soldiers  who 
participated  in  the  training  had  diverse  military  backgrounds  and  were  representative  of  the  chain 
of  command  within  an  Infantry  platoon,  from  the  platoon  leader  to  the  rifleman.  The  research 
also  showed  how  CBT  could  be  designed  to  incorporate  tactical  system  software  for  background 
instruction  and  demonstration  screens,  and  as  interactive  screens  for  performance  exercises, 
yielding  a  high-fidelity  training  environment. 

The  results  were  briefed  to  the  LW  Training  Working  Group,  the  Project  Manager- 
Soldier  Electronics/Land  Warrior  (PM-SE/LW),  and  the  LW  Manpower  and  Personnel 
(MANPRINT)  Working  Group  in  May,  June,  and  August  of  2001  respectively.  The  findings 
provided  the  PM-SE/LW  with  needed  information  on  how  computer-based  training  could  be 
incorporated  in  the  design  of  future  training  for  the  Land  Warrior  system. 

OTA  M.  SIMUTIS 
'Technical  Director 
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WORKING  MEMORY  AND  EXPLORATION  IN  TRAINING  THE 
KNOWLEDGE  AND  SKILLS  REQUIRED  BY  DIGITAL  SYSTEMS 


EXECUTIVE  SUMMARY 


Research  Requirement: 

Much  future  training  for  equipment  in  the  Army’s  digitized  forces  will  be  done  with 
some  form  of  interactive  multi-media  or  computer-based  training  (CBT).  Knowing  how  to 
develop  effective  CBT  for  the  diversity  of  soldiers  and  leaders  employing  these  digital  systems  is 
critical.  When  designing  CBT,  instructional  designers  make  important  conceptual  decisions  that 
influence  the  learning  process.  One  decision  is  how  much  material  to  include  in  a  lesson. 
Another  decision  is  the  extent  to  which  the  training  should  be  highly  structured  as  compared  to 
giving  individuals  the  flexibility  to  learn  on  their  ovm.  The  research  described  here  examined 
these  issues  within  the  context  of  prototype  software  for  the  future  Land  Warrior  system.  It 
provided  guidelines  and  insights  into  how  to  design  CBT  programs  that  will  be  successful  with 
the  diversity  of  soldiers  and  leaders  in  the  Army. 


Procedure: 

Soldiers,  a  total  of  168,  from  four  Infantry  courses  at  Fort  Benning,  GA  participated, 
including  those  in  basic  training,  noncommissioned  officer  courses,  and  the  basic  officer  course. 
Within  each  course,  soldiers  were  randomly  assigned  to  the  training  conditions.  All  training  was 
conducted  via  CBT;  there  was  no  instructor.  Soldiers  progressed  at  their  own  rate.  The 
experiment  had  two  phases.  In  the  first  phase,  two  variations  of  CBT  were  developed  to  train 
soldiers  to  learn  unique  codes  for  identifying  units  and  individuals  displayed  on  a  digitized  map. 
One  condition  placed  high  demands  on  soldiers’  working  memory  by  presenting  much 
information  before  soldiers  had  an  opportunity  to  apply  this  information.  The  other  condition 
placed  low  demands  on  working  memory,  by  breaking  up  the  lessons  and  exercises  into  smaller 
chunks.  In  the  second  phase  of  the  experiment,  soldiers  were  trained  on  map  skills.  Low  and 
high  demand  conditions  were  implemented  again,  and  an  exploratory  condition  was  added.  In 
the  exploratory  condition,  soldiers  were  informed  of  the  map  functions  they  had  to  learn,  but  not 
how  to  execute  the  required  steps. 


Findings: 


Consistent  differences  in  the  soldiers  from  the  different  courses  occurred  in  both 
experimental  phases,  with  officers  typically  achieving  the  highest  scores  in  the  shortest  amount 
of  time,  and  Infantry  trainees  scoring  the  lowest,  taking  the  most  time.  In  both  training  phases, 
differences  in  favor  of  the  low  working  memory  demand  condition  occurred  for  the  instructional 
segments  where  the  amount  of  information  presented  was  the  most  discrepant  between  the  high 
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and  low  demand  conditions.  On  the  map  final  exam,  soldiers  in  the  exploratory  condition  had 
the  lowest  scores,  with  those  in  the  low  demand  condition  achieving  the  highest  scores. 

However,  those  in  the  exploratory  condition  completed  their  training  and  testing  in  half  the  time 
required  in  the  low  and  high  working  memory  demand  conditions.  In  summary ,  the  low  demand 
condition  was  the  most  effective  for  code  and  map  skills. 


Utilization  of  Findings: 


Not  unexpectedly,  too  much  information  can  be  included  in  a  block  of  instruction.  But 
blocks  of  instruction  should  not  be  made  small  just  to  reduce  the  error  rate.  They  should  not 
fragment  content  or  ignore  the  need  to  consolidate  and  integrate  information.  The  exploratory 
mode  of  training  showed  potential,  particularly  for  the  interactive  skills  and  insights  needed  by 
soldiers  in  working  with  tactical  system  interfaces,  but  might  be  more  effective  when  combined 
with  some  traditional  modes  of  instruction.  The  experiments  showed  how  CBT  could  be 
designed  to  incorporate  tactical  system  software  as  background  instruction  and  demonstration 
screens,  and  as  interactive  screens  for  performance  exercises.  High-fidelity  training  is  a  positive 
by-product  of  this  technique.  Although  the  multi-media  instruction  was  effective,  the  medium 
per  se  is  not  an  automatic  panacea  for  training  the  digital  skills  required  of  soldiers  and  leaders. 
Measurement  techniques  and  procedures  are  needed  that  account  for  the  multiple,  yet  valid 
approaches  a  soldier  can  use  to  accomplish  a  task.  In  addition,  challenges  lie  in  developing 
problem-solving  scenarios  that  require  soldiers  to  go  beyond  demonstrating  technical  skill 
proficiency  to  demonstrating  effective  employment  of  digital  skills  in  combat-like  settings. 
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Working  Memory  and  Exploration  in  Training  the 
Knowledge  and  Skills  Required  by  Digital  Systems 


Introduction 

The  Army  is  now  automating  the  battlefield,  creating  what  is  known  as  digitized  forces. 
Computer  software  is  being  embedded  in  vehicles,  to  include  tanks,  command  and  control 
vehicles,  and  Infantry  fighting  vehicles  (General  Accounting  Office  [GAO],  2000).  In  addition, 
the  dismoimted  soldier  will  eventually  have  a  wearable  computer,  such  as  that  in  the  Land 
Warrior  (LW)  system  (Goodman,  1999).  Soldiers  and  leaders  must  use  this  software  to  execute 
tasks  such  as  sending  messages  and  orders,  and  creating  overlays;  tasks  which  were  formerly 
conducted  without  a  computer.  In  addition,  these  computer  technologies  give  soldiers  and 
leaders  additional  capabilities;  for  instance,  real-time  monitoring  of  the  location  and  movement 
of  units  using  digitized  maps.  In  this  report,  we  refer  to  this  equipment  as  “digital  systems,  and 
the  required  soldier  skills  as  “digital  skills.” 

This  large-scale  fielding  of  computer  technologies  raises  training  questions.  In  particular, 
we  need  to  know  how  to  best  train  the  diverse  population  of  soldiers  and  officers  who  employ 
these  systems.  One  approach  is  to  use  computers  as  a  means  of  delivering  training.  Much  is 
known  about  learning  and  cognition,  yet  questions  remain  regarding  designing  for  interactive 
multi-media  training  systems.  Issues  exist  about  how  tasks  “should  be  broken  down  into  practice 
activities,  what  activities  the  student  should  engage  in  first,  how  the  activities  should  be 
organized,  how  much  practice  the  student  needs  in  each  of  the  activities,  and  so  on.  Poor 
instruction  often  arises  from  the  guesswork  in  the  specification  of  these  details”  (Woolf  & 
Regian,  2000,  p.  353).  Theories  of  learning  and  training  provide  general  guidance  (Sanders, 
2001),  but  not  the  technical  details  required  for  the  designers  of  interactive  multi-media 
(computer-based)  training.  Empirical  work,  however,  can  address  these  unknowns,  and  reduce 
some  of  the  guesswork  involved  during  the  actual  creation  of  these  training  programs. 

The  experiment  described  in  this  report  focused  on  three  factors  that  can  influence  the 
design  of  computer-based  instruction  for  digital  systems. 

•  The  type  of  task  to  be  taught/trained:  Declarative  and  procedural  knowledge 

•  The  instructional  techniques:  The  volume  of  information  presented  before  the  user 
works  with  the  material  and  the  degree  of  instructional  guidance  provided 

•  Learner  characteristics:  Differences  in  Army  experience  and  knowledge 

Featured  in  many  of  the  Army’s  digital  systems  is  a  computerized  map  that  displays  the 
battlefield  locations  of  units  as  well  as  tactical  overlays  (Sanders,  2001).  This  is  true  of  the 
computer  display  in  the  Land  Warrior  (LW)  system.  A  unique  feature  of  the  LW  system  s  map 
is  that  it  must  show  individuals,  down  to  individual  squad  members,  as  well  as  units.  In  order  to 
use  the  map  effectively,  soldiers  must  know  the  codes  or  the  means  by  which  imits  and 
individuals  are  depicted.  A  challenge  with  the  instruction  described  here  was  to  develop  a 
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coding  scheme  that  would  uniquely  and  graphically  identify  individual  soldiers,  as  such  a 
scheme  does  not  currently  exist  within  the  Army. 

An  early  prototype  of  the  map  display  and  layout  proposed  for  the  LW  system  was 
simulated.  Some  modifications  were  made  to  the  prototype  for  purposes  of  the  experiment,  but 
basic  functions  such  as  panning  and  zooming,  displaying  individuals,  and  determining  distances 
on  the  map  were  retained,  and  are  similar  to  the  most  recent  version  of  the  system  software.  The 
experiment  provided  insights  into  the  particular  interface  that  was  simulated.  More  important,  it 
allowed  an  examination  of  the  generic  training  issues  mentioned  previously.  Participants  were 
soldiers  with  differing  Army  backgrounds  and  experience. 

Purpose 

The  training  experiment  had  several  purposes  that  have  general  applicability  to  many 
Army  digital  systems.  Participants  in  the  experiment  represented  the  soldiers  within  an  Infantry 
rifle  platoon  (young  soldiers,  noncommissioned  officers,  and  lieutenants).  Thus  the  research 
provided  information  on  how  soldiers  from  these  different  populations,  who  vary  considerably  in 
Army  experience  and  training,  would  learn  and  apply  the  concepts  taught  about  a  digital  map. 
Would  one  group  of  soldiers  learn  the  information  and  procedures  faster  and  to  a  higher  level  of 
proficiency  than  the  other  groups? 

Different  computer-based  training  procedures  were  also  compared.  In  reality,  many 
instructional  applications  are  characterized  by  intense  production  schedules  and  limited  resources 
(Lohr,  2000),  leaving  little  or  no  opportunity  for  research  on  comparing  the  effectiveness  of 
training  alternatives.  The  research  reported  here  examined  the  impact  of  the  demands  placed  on 
working  memory,  and  the  impact  of  an  exploratory  mode  of  training.  Soldiers  were  faced  with 
two  types  of  knowledge  requirements.  One  was  learning  the  unique  symbols  that  represented 
individuals  and  units  on  the  map.  The  second  was  learning  how  to  manipulate  the  map  interface 
to  solve  tactical  problems  representative  of  those  required  in  the  field.  With  both  types  of 
knowledge  requirements,  the  amount  of  information  presented  before  soldiers  had  an  opportunity 
to  apply  it  was  varied.  In  other  words,  some  soldiers  were  confronted  with  large  amounts  of 
information  prior  to  any  practical  exercises  (high  demands  on  working  memory).  For  others,  the 
same  information  was  divided  into  smaller  chunks  of  information  prior  to  practical  exercises 
(low  demands  on  working  memory).  How  much  information  could  be  presented  within  a 
training  session  without  overloading  the  soldiers? 

With  the  map  interface,  some  soldiers  used  an  exploratory  mode  of  learning  the  map 
functions.  Basically,  they  had  a  period  of  time  to  determine  how  each  function  worked,  without 
any  exercises  or  feedback  on  whether  they  had  learned  all  aspects  of  each  function.  Would 
soldiers  leam  the  map  functions  by  working  on  their  own  as  well  as  soldiers  given  formal 
instruction? 

A  corollary  to  the  above  issues  was  whether  the  different  soldier  populations  participating 
in  the  research  would  benefit  more  from  one  training  approach  than  another.  If  that  were  the 
case,  then  training  on  digital  systems  should  be  designed  to  adapt  to  these  differences. 
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Types  of  Knowledge 

Anderson  (1980)  distinguished  between  declarative  knowledge  (knowing  that),  and 
procedural  knowledge  (knowing  how).  Declarative  knowledge  reflects  the  facts,  concepts,  and 
principles  we  know;  while  procedural  knowledge  reflects  the  skills  we  know  how  to  perform,  to 
include  problem  solving,  reasoning,  and  means-ends  analysis.  For  Anderson,  procedural 
knowledge  or  cognitive  skill  learning  occurs  in  three  stages:  a  cognitive  stage  where  the 
description  of  the  procedure  is  learned,  an  associative  stage  where  the  method  for  performing  the 
skill  is  worked  out,  and  an  autonomous  stage  where  the  skill  becomes  more  and  more  rapid  and 
automatic.  Procedural  knowledge  can  have  both  motor  and  cognitive  components.  A  specific 
task,  obviously,  can  be  primarily  motor  (riding  a  bike)  or  primarily  cognitive  (programming  a 
computer).  Both  forms  of  knowledge  were  included  in  the  experiment. 

The  first  phase  of  the  experiment  involved  learning  a  coding  system  for  uniquely 
identifying  individuals  and  units;  that  is,  facts  or  declarative  knowledge.  This  system  combined 
standard  Army  symbols  with  the  battle  roster  (BR)  numbering  system.  The  second  phase 
focused  on  learning  the  procedural  skills  to  manipulate  a  digital  map  interface,  specifically  how 
to  manipulate  the  software  interface  to  solve  tactical  problems  in  locating  personnel  and  ground 
features,  and  to  determine  range  and  azimuth.  There  was  no  intent  to  train  these  procedural  skills 
to  an  autonomous  stage. 

Both  phases  compared  training  that  placed  differing  demands  on  working  memory.  With 
the  procedural  knowledge  in  the  map  phase,  it  was  also  possible  to  use  an  exploratory  training 
mode  where  there  was  minimal  instructional  guidance. 

Demands  on  Working  or  Short-term  Memory 

One  of  the  challenges  for  instructional  designers  of  computer-based  training  is  to  present 
new  information  so  it  can  be  kept  in  short-term  or  working  memory  for  a  sufficient  period  of 
time  to  be  processed  and  retained  and  retrieved  as  meaningful  material  in  long-term  memory. 

We  know  that  if  information  in  short-term  memory  becomes  inactive,  it  will  be  lost  unless  it  is 
also  in  long-term  memory.  As  pointed  out  by  Anderson  (1980),  it  is  helpful  to  “think  of  short¬ 
term  memory  as  a  working  memory  holding  only  knowledge  currently  in  use”  (p.  166).  But  our 
short-term  capacity  is  limited.  When  new  information  exceeds  the  short-term  memory 
capabilities  of  learners,  they  cannot  hold  it  in  short-term  memory  long  enough  for  it  to  be 
processed,  elaborated  upon,  and  transformed  into  long-term  memory.  “For  example,  if  we  lose 
track  of  the  telephone  number,  we  are  imable  to  dial  it  (Anderson  1980,  p.  166).” 

Andrews  and  Bell  (2000)  made  a  similar  point  in  discussing  simulation-based  military 
training.  “Structural  models  focus  on  identifying  the  mental  structures,  such  as  working,  short¬ 
term  and  long-term  memories  that  play  roles  in  information  storage  and  retrieval.  One  of  the 
difficulties  in  designing  virtual  learning  environments  is  understanding  how  much  new 
information  should  be  presented  at  one  simulation  session  before  learning  overload  occurs. 
Because  simulation-based  training  can  present  so  many  new  environmental  cues  at  once  to  the 
learner,  it  is  important  that  a  solid  understanding  of  learner’s  capacities  be  taken  into  account 
during  the  instructional  design  phase.  If  too  much  information  is  presented  too  quickly,  the 
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learner’s  cognitive  structures  can  be  overwhelmed  to  negative  effect  (p.  380).”  In  computer- 
based  training  experiments  on  combat  vehicle  identification,  Dyer,  Westergren,  Shorter,  and 
Brown  (1997)  found  that  when  too  many  vehicles  were  presented  (8  vehicles  each  with  8  aspect 
angles  for  a  total  of  64  images),  the  training  time  increased  and  soldiers  did  not  reach  criterion 
easily.  In  this  context,  the  recommendation  was  that  no  more  than  6  vehicles,  preferably  4  to  5, 
should  be  presented  in  a  training  session.  In  quite  a  different  context,  Trafton  and  Reiser  (as 
cited  in  Atkinson,  Derry,  Renkl  &  Wortham,  2000)  examined  the  effects  of  alternating  examples 
and  practice  to  a  blocked  condition  where  all  examples  were  presented  followed  by  all  practice 
conditions.  They  hypothesized  that  the  alternating  condition  allowed  the  participants  to  maintain 
an  example  in  working  memory  during  the  practice  session.  Results  showed  that  the  alternating 
example  and  practice  condition  was  more  effective. 

The  concept  of  meaningful  chunks.  Miller  (1956)  argued  in  his  classic  article  on  “7%e 
magical  number  seven,  plus  or  minus  two"  that  short-term  memory  was  limited  not  by  the 
number  of  physical  units  (letters  or  words),  but  by  the  number  of  meaningful  chunks.  The 
number  of  chunks  that  could  be  held  in  immediate  memory  was  postulated  to  be  seven,  plus  or 
minus  two.  This  span  of  immediate  memory  limits  the  amount  of  information  that  individuals 
can  receive,  process,  and  remember.  But  it  can  be  expanded  through  recoding  or  grouping 
information  into  larger  chunks  that  make  the  information  more  meaningful.  Miller 
acknowledged  in  that  article,  as  have  others  (Anderson,  1980),  that  it  is  difficult  to  define  what 
constitutes  a  chunk  of  information.  Nonetheless,  the  amount  of  information  we  can  hold  in 
short-term  memory  represents  a  limitation  on  our  mental  capacity. 

Since  Miller’s  (1956)  article,  more  is  known  now  about  short-term  memory,  and  slightly 
different  theoretical  frameworks  have  been  proposed.  The  extent  and  nature  of  recoding  depends 
on  previous  learning  (Anderson,  1980;  Baddeley,  1994;  Ye  &  Salvandy,  1994).  The  number  of 
chunks  is  dependent  on  the  degree  to  which  the  material  within  each  chunk  is  integrated  or 
related  (Baddeley,  1994).  Short-term  memory  paradigms  now  involve  retrieval  from  both  long¬ 
term  and  short-term  memory  (Shriffin  &  Nosofsky,  1994).  Many  other  factors  such  as  attention, 
interference  from  similar  inputs,  and  decision-making  affect  an  individual’s  short-term 
processing  of  information  (Baddeley,  1994;  Shriffin  &  Nosofsky,  1994). 

Applying  the  concept  of  chunks.  In  light  of  the  substantial  amount  of  research  on  short¬ 
term  memory  capacity,  the  many  factors  that  can  influence  processing  of  information,  and  the 
difficulty  in  defining  “chunks”  (Simon  1974),  it  may  seem  unrealistic,  and  it  probably  was 
simplistic,  to  try  to  design  experimental  conditions  that  specifically  varied  the  amount  of  material 
to  be  learned,  based  on  the  concept  of  chunks.  Nonetheless,  the  concept  was  used  to  help 
structure  the  experimental  conditions.  It  must  be  noted  that  most  of  the  material  presented  in  the 
experiment  was  unfamiliar  to  the  soldiers.  They  did  not  know  the  coding  scheme  we  presented; 
they  had  never  seen  the  map  interface  that  was  presented  to  them.  From  that  perspective,  they 
bar!  had  no  prior  opportunity  to  “recode”  this  material,  which  would  have  helped  them  process 
larger  amounts  of  information.  On  the  other  hand,  the  material  was  not  equivalent  to  the 
nonsense  syllables  and  strings  of  digits  or  words  used  in  much  of  the  work  on  short-term 
memory.  The  soldier’s  military  background  was  relevant  to  the  training  content.  The 
experiment  should  not  be  considered  as  a  formal  attempt  to  psychologically  define  or  measure 
chunks.  But  it  was  hoped  that  the  results  would  provide  some  guidelines,  but  certainly  not  a 
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formula,  for  instructional  designers  working  with  computer-based  training  programs,  when  faced 
with  a  similar  issue  of  how  much  information  to  present  in  a  “lesson”  and  when  it  is  assumed 
that  most  of  the  information  is  imfamiliar  to  the  learner. 

We  wanted  to  know  how  much  information  could  be  presented  within  a  training  lesson 
without  overloading  the  soldier,  without  giving  him  too  much  to  remember  and  process  at  once, 
and  therefore  creating  problems  in  learning.  Within  each  phase  of  the  experiment,  two  working 
memory  demand  conditions  were  generated,  based,  in  part,  on  the  concept  of  Miller  s  number 
seven.  The  Low  working  memory  demand  condition  (called  Low  Demand)  contained  lessons 
that  typically  presented  about  seven  or  fewer  new  concepts  or  codes  to  learn.  The  High  working 
memory  demand  condition  (called  High  Demand)  had  lessons  with  more  than  seven  concepts  or 
codes.  More  details  are  in  the  Method  section  on  how  we  “quantified”  the  number  of  chunks. 

Four  groups  of  soldiers  from  Fort  Benning,  GA,  with  differing  degrees  of  Army 
experience  and  knowledge,  participated  in  the  experiment.  The  most  inexperienced  were  the 
soldiers  from  Infantry  One  Station  Unit  Training  (OSUT).  These  were  young  soldiers  in  the 
midst  of  their  basic  and  advanced  individual  training,  who  had  not  been  assigned  to  an  Army 
unit.  Two  groups  of  noncommissioned  officers  (NCOs)  also  participated.  One  group  was 
enrolled  in  their  Basic  Noncommissioned  Officer  Course  (BNCOC);  the  other  in  their  Advanced 
Noncommissioned  Officer  Course  (ANCOC).  Lastly,  there  were  soldiers  enrolled  in  the  Infantry 
Officer  Basic  Course  (lOBC).  These  soldiers  could  have  been  commissioned  through  the  U.S. 
Army  Military  Academy,  through  the  Reserve  Officers’  Training  Corps  (ROTC),  or  through 
Officer  Candidate  School  (OCS).  OCS  commissions  NCOs  as  officers. 

Working  memory  is  influenced  by  the  meaningfulness  of  the  material  (Anderson,  1980). 
The  more  meaningful  the  material,  the  greater  the  memory  capacity.  The  more  familiar  soldiers 
are  with  weapon  symbols,  the  battle  roster  system,  or  the  Army’s  organizational  structure,  then 
the  more  meaningful  the  coding  scheme  and  the  fewer  demands  placed  on  their  working 
memory.  Of  the  four  soldier  populations  participating  in  the  research  (lOBC,  BNCOC,  ANCOC, 
and  OSUT),  the  OSUT  soldiers  have  the  least  Army  experience.  The  hierarchical  structure  of 
the  Army  and  even  the  structure  of  an  infantry  rifle  squad  are  not  ingrained  concepts  for  those  in 
OSUT.  On  the  other  hand,  ANCOC  soldiers  are  senior  noncommissioned  officers  with  typically 
12  years  in  the  Army.  Based  on  these  differences  in  Army  experience,  the  material  was  expected 
to  be  most  challenging  for  the  OSUT  soldiers. 

In  addition,  it  was  reasonable  to  expect  that  some  soldier  groups  would  be  affected  more 
strongly  than  others  by  the  High  Demand  condition.  Aptitude-treatment  interactions  have  long 
been  of  interest  to  researchers  (e.g.,  Berliner  &  Cahen,  1973;  Como  &  Snow,  1986).  With 
respect  to  cognitive  load  and  instructional  design,  Kalyuga,  Chandler,  and  Sweller  (1998) 
demonstrated  that  integrated  diagram  and  text  presentations  of  electrical  circuits  were  needed  to 
reduce  cognitive  load  for  inexperienced  individuals,  but  the  diagram  only  format  was  more 
effective  for  experienced  individuals. 

Aptitude-treatment  interactions  were  not  the  primary  focus  of  the  experiment.  However, 
as  the  soldiers  in  OSUT  had  the  least  military  knowledge,  it  was  expected  that  differences 
between  the  Low  and  High  working  memory  demand  conditions  would  be  greater  for  them  than 
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for  the  other  soldier  groups  (BNCOC,  ANCOC  and  lOBC).  The  soldiers’  knowledge  of  Army 
symbols,  the  battle  roster  numbering  system,  and  map  reading  was  assessed,  and  provided 
objective  indices  of  relevant  prior  knowledge.  Yet  not  all  dimensions  that  could  affect  the 
soldiers’  performance  (e.g.,  knowledge  of  the  structure  of  military  organizations)  were  assessed. 
Nor  was  there  any  attempt  to  directly  assess  differences  in  how  the  groups  structured  their 
military  knowledge  as  has  been  done  with  other  knowledge  domains  (e.g.,  computer 
programming.  Ye  &  Salvendy,  1994). 

Exploratory  Modes  of  Learning  and  Training  -  Problem-solving 

The  map  functions  allowed  comparison  with  another  approach  to  training.  Instead  of 
explaining  or  describing  how  the  functions  worked,  we  wondered  what  would  happen  if  we  let 
the  soldiers  discover  how  these  functions  worked  with  no  guidance,  instruction,  or  feedback.  In 
other  words,  treat  the  map  tasks  as  problem-solving  tasks.  Present  the  interface,  state  what  each 
function  does,  but  not  describe  the  procedures  involved.  Explore  -  figure  it  out  on  your  own 
with  no  hints  or  guidance  on  what  to  do.  The  goal  was  to  determine  how  each  function  worked. 
The  individual  worked  with  the  map  interface  until  personally  satisfied  that  he  understood  each 
function.  Then  he  progressed  to  the  map  final  exam. 

The  map  functions  in  the  experiment  did  not  require  great  insight  or  creativity,  although 
it  was  presumed  that  soldiers  had  the  basic  computer  skills  required  to  work  with  menus  and 
icons.  However,  even  when  a  computer  environment  is  relatively  simple,  the  level  of  complexity 
can  increase  significantly  if  the  user  makes  an  error,  venturing  down  an  incorrect  path.  Greif 
(1994)  suggested  that  the  complexity  of  a  computer  environment  is  not  a  stable  characteristic. 
Instead,  level  of  complexity  may  be  altered  by  user  error,  task  novelty,  and  slight  modifications 
in  system  configuration. 

There  were  several  reasons  for  having  an  exploratory  training  condition  for  the  map 
interface  (procedural  skills).  One,  it  is  often  stated  that  soldiers  can  and  will  figure  things  out  on 
their  own.  If  you  just  give  them  the  opportimity,  they  will  decipher  how  a  software  package 
works.  Two,  if  successful,  an  exploratory  mode  would  provide  more  training  flexibility  and  an 
alternative  that  might  work  very  well  for  some  soldiers.  Third,  research  on  exploratory  training 
had  shown  potentially  positive  and  negative  aspects,  which  were  important  to  investigate  in  the 
soldier  populations  of  interest.  Fourth,  computer  software  seemed  an  appropriate  content  domain 
for  exploratory  learning. 

Research  on  exploratory  learning  has  revealed  both  advantages  and  disadvantages  over 
traditional  training.  One  reported  advantage  is  improved  transfer  of  learning  (Carroll,  1997; 

Egan  &  Greeno,  1973;  Kamouri,  Kamouri,  &  Smith,  1986).  This  improvement  in  transfer  is 
believed  to  result  from  the  use  of  analogical  reasoning  and  hypothesis  testing  when  acquiring 
procedural  knowledge.  It  has  also  been  reported  that  exploratory  learning  can  be  accomplished 
in  the  same  or  less  amount  of  time  as  instruction-based  training  (Carroll,  1997;  Kamouri, 
Kamouri,  &  Smith,  1986). 

The  reported  drawbacks  of  exploratory  learning  include  so  called  “exploration  traps” 
(Payne  &  Howes,  1992)  where  learners  may  select  inefficient  methods  to  accomplish  a  goal,  or 


6 


fail  to  remember  the  procedures  they  used  to  complete  a  task.  Individuals  may  not  possess  the 
appropriate  information-seeking  skills  (Wallace,  Kupperman,  Krajcik,  &  Soloway,  2000).  Even 
when  students  had  to  use  web  technologies  to  solve  problems,  their  information-seeking 
behavior  was  simplistic  (Wallace  et  al.,  2000).  Although  using  the  web  was  enjoyable, 
information  seeking  evolved  into  a  search  to  find  a  good  Web  site.  Students  did  not  use  search 
engines  well  nor  relevant  information  they  found  during  the  search. 

In  addition,  individuals  who  lack  experience  with  a  task  may  not  possess  the 
metaknowledge  to  recognize  what  they  do  not  know.  It  has  been  reported  that  diversity  of 
experience  has  a  positive  influence  on  information  seeking  during  exploration  (Briggs,  1990). 
High  ability  individuals  may  do  well  in  an  exploratory  environment,  but  those  with  lower  ability 
apparently  require  more  learning  support  (Shute,  Lajoie,  &  Gluck,  2000).  Some  research  has 
addressed  methods  of  providing  a  “supportive”  exploratory  environment  in  which  potential 
“traps”  are  minimized  and  the  benefits  of  exploratory  learning  are  optimized  (DeMul  &  Van 
Oostendorp,  1996;  Trudel  &  Payne,  1995,  Van  Oostendorp  &  De  Mul,  1999).  For  example, 
Trudel  and  Payne  improved  exploratory  learning  of  a  computer  simulated  digital  watch  by 
restricting  the  number  of  keystrokes  allowed  during  a  learning  session. 

Individual  Differences  in  Exploration  Tendency 

Within  the  context  of  research  on  exploratory  behavior  and  curiosity,  computer  software 
systems  have  been  referred  to  as  the  “modem  labyrinths  for  humans,”  and  are  ideal  for  research 
on  exploratory  behavior  (Greif,  1994,  p.  287).  Research  in  this  area  (e.g.  Carroll,  1997;  Carroll, 
Mack,  Lewis,  Grischkowsky,  &  Robertson,  1985;  Greif,  1994;  Tmdel  &  Payne,  1995)  is  still 
relatively  young,  and  has  focused  upon  issues  related  to  software  design  (e.g.  the  complexity  of 
software  systems,  providing  help  features  to  facilitate  error  management),  and  instmctional 
design  methods  (such  as  minimal  manuals,  mode  restriction,  and  adaptive  task  complexity). 

Outside  the  realm  of  computer  research,  individual  differences  in  curiosity  and 
exploratory  behavior  have  been  studied  in  a  wide  variety  of  settings,  using  diverse  methods  and 
stimuli  (Henderson,  1994).  Much  of  this  research  has  been  conducted  among  children,  and  it  is 
widely  accepted  that  the  predisposition  to  explore  differs  among  children.  However,  research 
has  also  demonstrated  that  the  tendency  to  explore  is  influenced  by  setting  and  the  type  of 
exploratory  stimulus  involved.  For  example,  a  child’s  tendency  to  wander  and  explore  a  new 
physical  environment  does  not  necessarily  predict  a  tendency  to  explore  other  novel  stimuli,  or  a 
preference  for  complex  over  simple  stimuli  (Henderson,  1 994).  Research  on  individual 
differences  in  exploring  computer  systems  has  focused  primarily  on  the  interaction  between 
experience  (expertise)  and  characteristics  of  the  interface  such  as  complexity  (see  Greif,  1994) 
and  help  features  (e.g.,  De  Mul  &  Van  Oostendorp,  1996).  However,  little  attention  has  been 
given  to  the  question  of  whether  some  individuals  are  naturally  more  inclined  to  learn  their  way 
around  a  computer  system  through  exploration  of  the  interface. 

As  we  were  planning  the  present  study,  we  were  unable  to  find  a  measure  of  this 
tendency  to  bypass  manuals  and  instructions,  and  learn  through  active  exploration.  For  the 
remainder  of  this  report  we  will  refer  to  this  construct  as  a  tendency  to  work  independently. 
Therefore  a  scale  was  developed  to  assess  how  participants  approach  a  variety  of  tasks  for  which 


7 


it  would  be  possible  to  either  follow  step-by-step  instructions  or  plunge  right  in  and  attempt  the 
task  independent  of  directions.  Following  our  development  of  the  tendency  to  work 
independently  scale,  it  was  learned  that  a  similar  scale  has  been  developed  in  Dutch  by  Glasbeek 
(as  cited  by  DeMul  &  Van  Osstendorp,  1996),  who  found  that  individuals  with  an  exploratory 
learning  style  were  more  suited  for  minimal  manuals  than  traditional  manuals.  De  Mul  and  Van 
Oostendorp  (1996)  modified  and  administered  the  scale  in  an  experiment  of  “exploration- 
supportive”  facilities  in  a  computer  interface.  We  were  unable  to  obtain  a  copy  of  the  scale. 
However,  a  brief  description  in  De  Mul  and  Van  Oostendorp  indicates  that  our  approach  to 
assessing  this  construct  was  somewhat  different  from  theirs.  De  Mul  and  Van  Oostendorp 
presented  participants  v^th  statements  that  each  described  an  approach  to  learning  a  new 
apparatus,  and  had  them  rate  the  frequency  with  which  they  used  each  approach.  In  contrast,  we 
presented  soldiers  with  fifteen  activities  and  asked  them  to  indicate  which  of  three  learning 
approaches  they  would  use  for  each  task.  A  detailed  description  of  our  scale  is  in  the  method 
section. 

De  Mul  and  Van  Oostendorp  (1996)  found  no  relationship  between  scores  on  their 
“learning  style”  scale  and  performance  with  a  novel  computer  interface.  However,  the 
researchers  indicated  that  the  interface  may  have  been  too  simple  for  differences  to  emerge,  and 
observed  that  they  had  a  ceiling  effect  in  which  participants  essentially  mastered  the  interface 
after  performing  a  few  tasks.  In  the  present  experiment,  it  was  expected  that  a  tendency  to  work 
independently  would  be  related  to  success  at  mastering  the  interface  for  the  soldiers  in  the 
Exploratory  map  learning  condition.  The  relationships  between  tendency  to  work  independently, 
time  spent  exploring,  and  scores  on  the  map  final  exam  were  examined. 

Computer-Based  Training  (CBT) 

All  training  in  the  experiment  was  computer-based,  as  it  was  envisioned  that  this  form  of 
training  will  eventually  be  common  with  the  Army’s  digital  systems.  Research  has  shown  that 
CBT  can  be  more  efficient  and  effective  than  traditional  classroom  instruction,  and  that  it  adapts 
well  to  differences  in  the  learner’s  rate  and  level  of  competency  (Fletcher,  2000;  Gibbons  & 
Fairweather,  2000).  Also  it  was  of  interest  to  know  the  challenges  posed  in  writing  courseware 
for  a  digital  environment. 

Of  particular  interest  was  prior  training  research  with  CBT  on  the  Army’s  digital  systems. 
Sanders’  (2001)  identified  principles  of  learning  from  behaviorist,  cognitive  and  constructivist 
theories  that  could  be  applied  to  designing  computer-based  training  for  command,  control, 
communications,  computers,  and  intelligence  (C4I)  systems,  such  as  the  Force  XXI  Battle 
Command  Brigade  and  Below  (FBCB2)  system.  In  a  follow-on  effort,  Deatz  and  Campbell 
(2001)  demonstrated  how  one  might  incorporate  some  of  these  cognitive  principles  (advance 
organizers,  chunking,  frames,  rehearsal,  etc.)  within  CBT.  But  this  was  only  a  demonstration, 
not  a  training  program.  Sanders  (2001)  referred  to  a  web  site  that  provided  training  on  FBCB2. 
However,  in  reality  this  medium  was  similar  to  an  electronic  version  of  a  technical  manual.  It 
was  not  an  interactive  training  medium  where  specific  skills  were  taught,  practiced  and  tested 
within  an  operational  context  (hypothetical  real-world  missions  and  situations),  and  where 
transfer  to  new  situations  was  examined.  Schaab  and  Moses  (2001)  cited  observations  of 
soldiers  using  and  training  with  the  All  Source  Analysis  System  (ASAS),  a  digital  system  for 
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military  intelligence  personnel,  but  CBT  was  not  used  in  that  training.  In  summary,  we  did  not 
find  any  research  on  the  training  effectiveness  of  different  CBT  approaches  for  training  digital 
skills  with  the  Army’s  “new”  digital  systems. 

Also  of  interest  was  whether  any  prior  CBT  research  had  examined  the  effects  of 
discovery  learning  or  the  role  of  working  memory.  Some  of  the  research  cited  earlier  in  this 
report  did  address  CBT-related  issues  (e.g.,  Trudel  &  Payne,  1995),  but  most  did  not.  Trudel  and 
Payne  simulated  a  commercial  digital  watch.  They  found  that  imposing  a  keystroke  limit  during 
the  learning  process  or  forcing  subjects  to  explore  one  mode  at  a  time  were  better  training 
techniques  than  allowing  subjects  to  explore  the  simulated  watch  on  their  own  without  any 
constraints.  Some  additional  examples  of  research  that  examined  CBT  in  conjunction  with 
working  memory  are  cited  below. 

Chandler  and  Sweller  (1996)  did  not  examine  CBT  per  se,  but  did  focus  on  the  demands 
on  working  memory  when  learning  a  computer-aided  design  software  package.  They  examined 
the  impact  of  the  design  and  use  of  computer  manuals  on  cognitive  load.  When  high 
interactivity  with  the  computer  was  required,  a  manual  format  that  required  holding  text 
information  in  working  memory  while  searching  and  working  with  the  computer  screen  was  less 
effective  than  a  manual-only  approach.  When  low  interactivity  with  the  computer  was  required, 
then  the  format  of  the  training  manual  did  not  impact  learning.  Shute  (as  cited  in  Shute  et  al., 
2000)  found  an  aptitude-treatment  interaction  involving  working  memory  capacity  in  the 
learning  of  a  flight-engineering  tutor.  Individuals  with  high  working  memory  capacity  and  high 
general  knowledge  consistently  performed  well;  those  low  on  both  dimensions  performed  poorly. 
The  interaction  effect  occiured  with  those  high  on  one  dimension  and  low  on  the  other. 
Specifically,  individuals  with  low  working  memory  capacity  but  much  general  knowledge 
benefited  fi-om  being  assigned  to  an  abbreviated  training  condition,  rather  than  an  extended 
period  of  training.  Apparently,  the  extended  condition  led  to  either  boredom  or  fatigue.  Those 
with  high  working  memory  capacity  but  low  general  knowledge  performed  best  in  the  extended 
practice  condition,  as  they  were  able  to  benefit  from  the  extra  practice.  In  examining  the  use  of 
computer  software  for  a  shopping  task  (determining  which  store  had  the  best  prices  for  differing 
numbers  of  items),  Wright  and  Lickorish  (1994)  found  that  as  the  number  of  steps  needed  to 
solve  a  problem  increased  (more  demands  on  working  memory),  the  more  likely  participants 
were  to  use  various  memory  aids  available  to  them.  However,  which  memory  or  navigational 
aids  participants  were  most  likely  to  use  was  not  predictable.  The  authors  concluded  that  the 
“cognitive  consequences  of  moving  through  texts  displayed  on  computer  screens  are  far  from 
trivial”  (p.  1003),  whereas  the  cognitive  consequences  of  page  turning  with  books  and  manuals 
seem  trivial. 


Method 


Participants 

Soldiers  from  four  U.S.  Army  Infantry  School  courses  participated.  These  courses  were 
the  lOBC,  BNCOC,  ANCOC,  and  Infantry  OSUT.  The  Infantry  OSUT  soldiers  were  in  the  1 1th 
week  of  OSUT,  which  is  a  14-  to  16-week  course.  Both  BNCOC  and  ANCOC  soldiers  were 
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mid-way  through  their  courses,  which  are  7  weeks  and  10  weeks,  respectively,  in  length.  All 
Infantry  officers,  regardless  of  commissioning  source,  attend  the  lOBC,  a  1 6-week  course.  Some 
officers  had  graduated  from  lOBC;  others  were  waiting  to  attend.  These  differences  did  not 
affect  the  results. 

The  experimental  design  called  for  48  soldiers  from  each  course.  The  actual  number  of 
participants  was  48  each  from  OSUT  and  BNCOC,  42  from  lOBC,  and  30  from  ANCOC. 

There  were  considerable  differences  among  the  soldiers  in  the  courses  as  reflected  in 
their  rank,  time  in  military  service,  and  age  (see  Appendix  C).  All  Infantry  OSUT  soldiers  held 
the  rank  of  private.  All  lOBC  soldiers  held  the  rank  of  2d  lieutenant.  All  but  one  of  the  ANCOC 
soldiers  held  the  rank  of  sergeant  first  class.  BNCOC  soldiers  held  the  ranks  of  sergeant  (31%) 
and  staff  sergeant  (69%).  ANCOC  soldiers  had  been  in  the  Army  a  mean  of  1 1.51  years  (SD  = 
2.21);  BNCOC  a  mean  of  7.47  years  (SD  =  2.60),  and  lOBC  a  mean  of  1.93  years  (SD  =  2.29). 
The  OSUT  soldiers  were  not  asked  about  their  prior  service  time,  as  they  were  trainees. 

However,  they  were  questioned  about  their  educational  background.  Over  half  (65%)  had  a  high 
school  education,  one  soldier  had  a  general  education  development  (GED)  diploma,  and  the 
remaining  33%  had  some  education  beyond  high  school  (college  or  technical  school).  The  mean 
ages  of  the  groups  differed  significantly,  F(3, 163)  =  137.59,/?  =  .0000.  OSUT  soldiers  were  the 
youngest  (M=  19.58,  SD  =  2.36).  lOBC  was  the  next  oldest  group  (M=  23.13,  SD  =  1.92). 
BNCOC  was  the  next  oldest  group  (M=  21  .IS,  SD  =  3.31).  ANCOC  was  the  oldest  group  (M= 
31.59,  SD  =  3.40).  Considering  all  soldiers,  the  youngest  was  17;  the  oldest  was  39. 

Procedure 

Soldiers  within  each  of  the  four  courses  were  randomly  assigned  to  one  of  four 
experimental  training  conditions.  The  experiment  was  executed  four  times;  once  for  each  group 
of  soldiers.  Three  multi-media  rooms  in  the  Infantry  School  were  used.  Each  room  had  16 
computers,  allowing  a  maximum  of  48  soldiers  to  participate  at  a  given  time  (see  Figure  1).  Four 
hours  were  allowed  for  the  experiment.  Software  for  each  of  the  experimental  conditions  had 
been  loaded  on  the  computers  prior  to  beginning  the  experiment. 

Before  starting,  soldiers  were  briefed  on  the  scope  of  the  experiment.  They  were 
encouraged  to  take  a  break  after  completing  the  code  exam  and  before  beginning  the  map 
training.  They  could  take  other  breaks  if  desired,  but  only  after  an  exercise  or  lesson. 
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Figure  1.  Multi-media  classrooms  where  the  experiment  was  conducted. 


Experimental  Design 

The  experimental  design  is  summarized  in  Table  1 .  Phase  I  was  individual  and  unit  code 
training.  Phase  II  was  map  interface  training.  The  same  information  and  concepts  were  taught 
and  applied  in  the  High  and  Low  Demand  conditions.  The  only  difference  in  the  High  and  Low 
Demand  conditions  was  how  this  information  was  partitioned  across  the  lessons  and  exercises. 
The  content  of  the  lessons  and  exercises  was  identical.  Table  1  also  shows  that  the  Exploratory 
condition  was  only  in  the  Map  phase. 

Table  1 

Experimental  Design  for  the  Code  and  Map  Training  Conditions 


Experimental  Conditions 


Low  Demand  Code 
&  Low  Demand  Map 

High  Demand  Code  & 
High  Demand  Map 

J- 

Low  Demand  Code  & 
Exploratory  Map 

J. 

High  Demand  Code  & 
Exploratory  Map 

J. 

Code  Lessons  and  Exercises:  Phase  I 


Two  common  blocks  of  instruction 


Six  blocks  of 

Three  blocks  of 

Six  blocks  of 

Three  blocks  of 

instruction 

instruction 

instruction 

instruction 

1  Final  Exam  on  Codes 

Map  Lessons  and  Exercises:  Phase  11 


Four  blocks  of 

One  block  of 

Brief  explanation  of 

Brief  explanation  of 

instruction 

instruction 

each  function;  no 

each  function;  no 

instruction 

instruction 

Final  Exam  on  Map  Functions 


Note.  Each  block  of  instruction  concluded  with  practice  exercises.  There  were  no  exercises  for 
the  instruction  on  range  and  azimuth  map  functions. 
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The  coding  system.  In  designing  the  experimental  conditions  for  training  individual  and 
unit  codes,  we  viewed  the  specific  code  required  to  uniquely  identify  each  position  as  a  chunk  of 
information.  Admittedly,  this  was  a  subjective  decision,  as  the  codes  were  not  independent  of 
each  other.  But  this  provided  a  basis  for  partitioning  the  instructional  materials,  as  there  are  no 
firm  guidelines  for  determining  “chunks”  of  meaningful  information.  Compared  to  many 
memory  experiments,  the  tasks  were  not  simply  memory  tasks.  Specific  concepts  and  mles 
provided  a  scheme  and  format  for  deciphering  the  codes.  In  addition,  following  the  presentation 
of  each  block  of  information,  training  exercises  were  provided  that  allowed  the  soldiers  to  work 
with  the  particular  codes  that  had  just  been  presented. 

Soldiers  had  to  learn  a  coding  system  for  every  individual  within  an  Infantry  platoon  plus 
key  leaders  at  the  company  level.  The  code  or  symbol  system  involved  combining  the  symbols 
for  the  primary  weapon  (Department  of  the  Army  [DA],  1997)  carried  by  individuals  in  each 
duty  position  with  what  is  known  as  the  battle  roster  (BR)  numbering  system  (DA,  1994).  These 
weapon  symbol/BR  combinations  were  developed  specifically  for  the  research.  Figure  2 
provides  examples  of  these  unique  codes.  These  codes  could  be  considered  facts  or  declarative 
knowledge  (Anderson,  1980). 

Five  weapon  symbols  and  three  unit  symbols  were  taught.  The  BR  numbering  system 
was  explained.  The  system  provides  for  a  five-character  alphanumeric  code  that  uniquely 
identifies  individual  duty  positions  from  the  rifleman  to  the  battalion  commander.  The  training 
focused  on  selected  positions  within  the  battalion.  Soldiers  learned  the  BR  numbers  for  all  nine 
members  of  a  rifle  squad,  all  nine  members  of  the  weapons  squad,  the  platoon  leadership  (3  duty 
positions),  plus  the  company  leadership  (3  duty  positions).  In  addition,  the  coding  scheme 
covered  unique  codes  for  units  at  the  squad,  platoon,  and  company  level.  Consequently,  for  a 
battalion,  soldiers  were  taught  to  identify  the  code  for  any  company  commander,  executive 
officer,  first  sergeant,  platoon  leader,  platoon  sergeant,  squad  leader,  rifleman,  etc. 

The  BR  roster  number  itself  provides  a  unique  code  for  each  duty  position.  We  added  the 
weapon  symbol  to  help  soldiers  distinguish  among  the  nine  individual  squad  members.  For 
example,  by  presenting  the  M203  weapon  symbol,  the  soldier  immediately  knew  the  individual 
was  a  grenadier,  even  though  he  may  have  forgotten  the  unique  number  for  a  grenadier  (a  3  or  a 
7).  A  description  of  the  battle  roster  system  and  the  weapon  symbols  used  in  the  training  is  in 
Appendix  A. 
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Weapon  Symbol  Plus  Battle  Roster  Number  Equals  Identifying  Code 


V 

AN1110 

I 

AN1110 

Symbol  for 

M16  rifle,  M4 
Carbine 

Battle  Roster 
Number 

Code  for  squad  leader 
from  1st  squad,  1st 
platoon,  A  Company 

Symbol  for 
M203  grenade 
Launder 


Battle  Roster 
Number 


AN3223 

1 

AN3223 

Code  for  grenadier 
from  A  team,  2d  squad, 
2d  platoon,  C  Company 


,  - 1 

Symbol  for 

Rifle  Squad 

Battle  Roster 
Number 

Code  for  3d  squad,  3d 
platoon,  C  Company 

Figure  2.  Examples  of  individual  and  unit  codes  resulting  from  combining  symbols  with  the 
battle  roster  (BR)  numbering  system.  [BR:  first  two  letters  designate  the  battalion,  1st  number  the 
company,  2d  number  the  platoon,  3d  number  the  squad,  and  last  number  the  individual  within  the  squad. 
This  last  number  also  indicates  whether  the  soldier  is  in  A  or  B  team.  The  first  code  shown  above  is  for  a 
squad  leader.  A  zero  (0)  in  the  last  position  with  non-zeros  in  the  preceding  positions  indicates  a  squad 
leader.  He  carries  an  Ml  6  rifle  or  M4  carbine.  The  second  code  is  for  a  grenadier.  The  third  code 
illustrates  a  unit  -  the  squad.  The  BR  number  is  truncated  at  the  squad  position.] 


Table  2  compares  the  Low  and  High  Demand  conditions  by  lesson.  A  chunk  was  defined 
as  a  unique  weapon  or  unit  symbol  (graphic)  or  as  a  unique  number  or  position  Aat  defined  a 
duty  position.  Chunks,  for  purposes  of  the  research,  represented  uniquely  identifiable  duty 
positions  or  units.  The  number  of  chunks  for  each  lesson  is  depicted.  Exercises  followed  each 
lesson.  For  example,  with  the  Low  Demand  condition,  lesson  C  was  presented  followed  by  the 
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exercises  for  C.  Then  lesson  D  was  presented  followed  by  the  exercises  for  D.  However,  with 
the  High  Demand  condition,  lessons  C  and  D  were  presented  first,  followed  by  exercises  C  and 
D.  Under  the  Low  Demand  condition,  less  information  was  presented  before  soldiers  worked  on 
exercises  involving  application  of  or  memory  for  the  information  in  the  lessons. 

Table  2 

Operational  Definitions  of  Chunks  of  Information  for  the  Code  Training  Conditions 


Low  Demand 

High  Demand 

Lesson  w/ 
Exercise 

#  Chunks 

Content  Description 

Lesson  w/ 
Exercise 

#  Chunks 

A 

8 

5  weapon  symbols  and  3  unit  symbols. 

A 

8 

B 

5 

The  5  positions  in  the  BR  numbering  system. 

B 

5 

C 

5 

The  5  numbers  for  company  BR  position. 

13 

D 

8 

The  4  numbers  for  platoon  position  and  4 
numbers  for  squad  position. 

C&D 

E 

9 

The  9  numbers  for  individuals  in  rifle  squad. 

E&F 

18 

F 

9 

The  9  numbers  for  individuals  in  weapons 
squad. 

G 

6  ! 

The  3  key  leaders  at  both  company  and 
platoon  echelons. 

G&H 

9 

H 

3 

BR  number  for  3  echelons  (units). 

Differences  in  the  High  and  Low  Demand  conditions  started  at  Lesson  C  (see  Table  2  and 
Figure  3).  As  indicated  in  Table  2,  the  information  presented  in  some  High  Demand  conditions 
was  double  that  presented  in  the  Low  Demand  conditions. 

A  summary  of  each  lesson  follows.  For  continuity,  the  abbreviations  in  the  summary 
titles  below  correspond  to  the  abbreviations  in  Figure  3. 

•  Lesson  A  -Symbols  (8  chunks):  Five  weapon  symbols  (Ml 6  rifle/M4  carbine,  M203  grenade 
launcher,  M249  squad  automatic  weapon,  M240B  machine  gun,  and  Javelin  antitank 
weapon)  and  three  unit  symbols  (squad,  platoon,  and  company)  were  presented.  Except  for 
the  M249  squad  automatic  weapon  (SA^,  all  symbols  were  the  same  as  those  in  the 
doctrinal  manual  on  symbols  FM  101-5-1  (DA,  1997).  As  there  is  no  unique  symbol  for  the 
SAW,  one  was  generated  for  that  purpose.  Soldiers  were  told  this  was  not  an  approved  Army 
symbol,  and  was  being  used  strictly  for  the  instruction  they  were  receiving. 

•  Lesson  B  -  BR  (5  chunks).  The  concept  of  the  Battle  Roster  (BR)  Numbering  System  was 
presented  with  an  explanation  of  the  five  positions  referenced  in  the  letter/number  string  that 
uniquely  defines  individual  duty  positions.  The  battalion  has  a  two-letter  code;  followed  by  a 
company  single  number  code;  a  platoon/section  number  code;  a  squad  number  code;  and  an 
individual  soldier  number  code. 
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Figure  3.  Instructional  sequence  for  code  training  conditions. 

•  Lesson  C  -  Co  BR  (5  chunks).  The  five  numbers  used  to  identify  companies  -within  a 
maneuver  battalion. 

•  Lesson  D-  Pit  Sd  BR  (8  chunks).  The  four  numbers  used  to  identify  platoons  were  presented 
first,  followed  by  the  four  numbers  used  to  identify  squads. 

•  Lesson  E  -  Rifle  (9  chunks).  The  numbers  assigned  to  individuals  within  a  rifle  squad  (0-8) 
were  presented  and  defined.  These  individuals  were  divided  by  fire  team  as  well.  At  this 
point,  the  weapon  symbol  was  combined  with  the  BR  number,  as  shown  in  Figure  2 
previously.  Soldiers  were  able  to  self-test  themselves  with  a  sample  of  codes  before 
proceeding  to  the  exercise. 

•  Lesson  F  -  Weapon  (9  chunks).  The  numbers  assigned  to  individuals  within  the  weapons 
squad  (0-8)  were  presented  and  defined.  The  weapon  symbol  was  combined  with  the  BR 
number,  as  shown  in  Figure  2.  Soldiers  were  able  to  self-test  themselves  with  a  sample  of 
codes  before  proceeding  to  the  exercise. 

•  Lesson  G  -  Leader  (6  chunks).  The  codes  for  three  key  leaders  at  both  the  company  and 
platoon  levels  were  presented.  These  were  the  company  commander,  executive  officer,  first 


sergeant,  platoon  leader,  platoon  sergeant,  and  radio-telephone  operator  (RTO).  The  weapon 
symbol  was  combined  with  the  BR  number  (all  leaders  had  the  rifle). 

•  Lesson  H  -  Unit  (3  chunks).  The  final  lesson  focused  on  codes  for  units,  specifically  the  rifle 
company,  platoon,  and  squad.  Although  there  are  official  symbols  for  each  of  these  types  of 
units  (presented  in  Lesson  A),  there  is  an  official  BR  type  number  as  well.  For  purposes  of 
the  training,  we  simply  truncated  the  BR  number  at  the  appropriate  point  to  reflect  the  unit. 
That  is  the  squad  BR  number  omitted  the  individual  position  (the  last  number).  The  platoon 
BR  number  omitted  squad  and  individual  numbers.  The  company  BR  number  omitted  the 
platoon,  squad  and  individual  numbers,  consisting  of  only  the  battalion  letters  and  the 
company  number. 

If  a  soldier  scored  lower  than  80%  on  any  code  exercise,  he  was  given  remedial  training 
on  the  corresponding  lesson.  He  then  retook  the  exercise. 

Map  functions.  A  total  of  seven  map  functions  was  taught;  zooming  in  and  out,  panning, 
finding  oneself,  finding  others,  displaying  others,  and  range/azimuth  determination.  Figure  4 
illustrates  the  map  display.  With  the  exception  of  range/azimuth,  all  functions  were  on  the  top 
toolbar.  Range/azimuth  was  at  the  bottom  of  the  display.  The  map  tasks  reflected  procedural 
knowledge  rather  than  the  declarative  knowledge  of  the  coding  system. 


^  LandWarrior  -  Training  Experiment 


[Mix 


*  Q,  %  ^1  © 

Out  In  Pan  Targ^ 


ME 

FmdMe  RndX 


OFF 

I  Lajiers  Draw  Lab^ 


Figure  4.  Map  display.  [Functions  included  in  the  experiment  have  a  thick  border.] 
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In  the  Low  Demand  map  condition,  the  functions  were  divided  into  the  four  blocks  of 
instruction  listed  here. 

Zoom  in  and  out,  pan,  and  find  oneself 
Find  others  (called  Find  X) 

Display  others 

Determine  range  and  azimuth 

In  the  Low  Demand  condition,  exercises  followed  each  block  except  for  range  and  azimuth.  In 
contrast,  in  the  High  Demand  map  condition,  instruction  on  all  functions  was  presented  first,  and 
then  individuals  were  allowed  to  practice  at  the  end  of  this  presentation.  Again,  there  were  no 
exercises  for  range  and  azimuth. 

It  was  harder  to  define  “chunks  of  information”  with  the  map  functions  than  with  codes. 
Clearly  distinguishable  units  or  entities  were  harder  to  define,  as  some  procedural  steps  were 
self-  cueing.  Chunks  primarily  reflected  the  number  of  functions  and  the  number  of  steps  within 
each  function  (see  Table  3).  The  button/function  itself  was  always  counted  as  one  chunk  of 
information  as  the  soldier  had  to  discriminate  it  from  the  other  buttons/functions  on  the  map 
display. 

Table  3 

Operational  Definitions  of  Chunks  of  Information  for  the  Map  Training  Conditions 


Low  Demand 

Content  Description 

High  Demand 

Lesson  w/ 
Exercise 

# 

Chunks 

Lesson  w/ 
Exercise 

# 

Chunks 

I 

6 

Zoom  in,  zoom  out,  pan,  and  find  oneself 
functions.  Two  functions  had  one  step;  two  had 
three  steps. 

I,  J,  K,  & 
M 

26 

J 

6 

Find  X  function,  with  5  types  of  menu 
selections:  find  unit,  go  to  lower  echelon,  go  to 
higher  echelon,  find  individual  soldiers,  &  find 
leaders  at  specific  echelon. 

K 

9 

Display  function.  It  triggered  3  menu 
selections;  with  3  ways  of  working  within  each 
menu,  2  unique  selections  within  two  of  the 
menus,  plus  a  set  of  procedures  covered  in 

Lesson  J. 

M 

5 

Range/azimuth  fimction:  2  clicks  to  establish 
points  on  map,  where  to  click  for  accuracy  and 
how  to  Ccincel. 

As  shown  in  Table  3,  the  information  presented  in  the  High  Demand  condition  was 
greater  than  that  in  each  Low  Demand  condition.  Additional  information  on  each  lesson  follows. 
The  lesson  summaries  describe  the  map  training  conditions.  The  lesson  titles  are  consistent  with 
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the  abbreviations  in  Figure  5.  Figure  5  graphically  illustrates  the  map  training  and  testing 
sequence  for  all  conditions. 

•  Lesson  I  -  Zoom  Pan  (6  chunks).  Four  buttons  or  functions  were  presented  (zoom  in,  zoom 
out,  pan,  and  find  oneself).  Two  (zoom  out  and  find  oneself)  were  only  one-step  (e.g.,  click 
on  the  button  and  the  map  zooms  out).  Both  the  zoom  in  and  pan  buttons  had  two  additional 
steps.  After  clicking  on  the  button  you  then  had  to  click  on  the  map  to  completely  activate 
the  function.  To  deactivate  the  function,  you  had  to  click  on  the  button  again. 

•  Lesson  J  -  Find  X  (6  chunks).  The  lesson  was  on  how  to  find  others  (units  and  individuals), 
called  find  X.  Only  one  entity  (unit  or  individual)  could  be  displayed  at  a  time  with  this 
function.  Only  one  button  was  used,  but  it  triggered  a  series  of  embedded  menu  selections. 

In  this  process,  the  soldier  had  to  learn  five  additional  critical  elements  (find  unit,  go  to  lower 
echelon,  go  to  higher  echelon,  find  individual  soldiers,  find  leaders  at  a  specific  echelon). 

•  Lesson  K  -  Display  (9  chunks).  The  lesson  was  on  how  to  display  units  and  individuals.  The 
function  was  designed  as  a  pre-planning  tool.  Individuals  could  decide  who  they  wanted  to 
display  on  their  map  during  a  mission,  and  then  make  these  selections  during  mission 
preparation.  Only  one  button  was  used,  but  it  triggered  three  menu  selections  (company 
menu,  platoon  submenu,  and  squad  submenu).  In  determining  the  number  of  chunks  of 
information,  we  counted  each  menu  as  one  chunk  of  information  (3  chunks).  Three  menu 
elements  were  similar  to  the  find  X  (Lesson  J),  and  therefore  were  counted  as  one  chunk  of 
information.  These  were  display  unit,  go  to  upper  echelon,  and  go  to  lower  echelon.  Within 
two  of  the  three  menus  (company  and  platoon),  there  were  two  new  categories  of  menu 
selections:  one  led  to  a  further  breakdovm  of  the  unit  and  the  other  led  to  a  block  of  “all” 
selections.  For  example,  with  the  platoon  submenu,  further  breakdowns  could  be  found  for 
Co  Hqs,  1st  platoon,  2d  platoon,  and  3d  platoon.  The  “all”  selections  within  the  platoon 
submenu  were:  all  platoons,  all  platoon  leaders,  all  platoon  sergeants,  and  all  squads.  So 
within  the  company  menu  and  the  platoon  submenu,  each  of  these  two  sections  was 
considered  a  critical  chunk  of  information,  for  an  additional  4  chunks  of  information.  (The 
labels  themselves  were  clear,  once  you  understood  the  concept).  Based  on  this  rationale,  the 
total  number  of  chunks  was  9.  The  rationale  for  defining  the  number  of  chunks  for  this  type 
of  task  was  consistent,  but  somewhat  subjective. 

•  Lesson  M  -  Rg  Azm  (5  chunks).  The  last  instruction  was  on  determining  range  and  azimuth. 
Again,  there  was  a  single  button,  two  clicks  were  required  to  establish  the  points  or 
units/individuals  on  the  map,  soldiers  had  to  know  where  to  click  to  get  an  accurate  reading, 
and  another  click  was  required  to  cancel  the  function. 

The  menus  in  the  find  X  and  display  functions  were  hidden  from  the  soldier’s  view,  so 
the  sequence  of  selections  was  not  apparent  from  the  map  display  (see  Figure  4).  The  instruction 
within  the  Low  and  High  Demand  lessons  revealed  the  various  menu  contingencies  to  the 
soldiers  (see  Appendix  D  for  illustrations).  Thus  soldiers  were  formally  taught  and  shown  the 
different  paths  available  to  them  with  these  functions.  This  technique  was  found  to  be 
particularly  beneficial  for  individuals  with  low  spatial  abilities,  although  the  technique  also 
helped  those  with  high  spatial  abilities  (Alonso  &  Norman,  1998). 
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Figure  5.  Instructional  sequence  for  map  training  conditions.  [The  E  refers  to  an  exercise.] 


No  remediation  was  provided  with  the  Low  and  High  Demand  map  conditions.  If 
soldiers  scored  less  than  80%,  they  proceeded  to  the  next  lesson  or  exercise.  Soldiers  were  told, 
however,  that  the  desired  score  was  at  least  80%.  There  were  two  reasons  for  no  remediation. 
First,  there  was  no  requirement  for  soldiers  to  learn  the  map  functions  as  a  prerequisite  for 
performing  a  later  task,  as  was  the  case  with  symbols  and  BR.  Knowledge  of  symbols  and  BRs 
was  essential  to  successfully  completing  the  map  tasks.  Second,  experimental  time  was  limited. 
There  was  a  high  probability  that  some  soldiers  would  not  have  completed  the  experiment  if 
remediation  had  been  required.  The  exercise  scores  were  examined,  however,  for  the  numbers  of 
soldiers  who  achieved  the  80%  desired  criterion. 

For  the  Exploratory  map  condition,  soldiers  were  given  minimal  guidance  and 
information.  They  were  informed  of  the  seven  functions  they  had  to  learn  through  exploration, 
shown  the  location  of  each  button  on  the  map,  and  told  the  purpose  of  each  button.  There  was  no 
instruction  on  how  these  functions  or  the  map  display  worked.  They  were  given  a  set  of  tasks 
they  could  try  to  execute  during  their  exploration,  but  were  not  required  to  do  these  tasks.  They 
had  60  minutes  to  work  with  the  map.  However,  they  could  progress  to  the  final  exam  whenever 
they  felt  confident  they  knew  how  each  button/fimction  worked.  The  soldiers  in  this  condition 
never  had  any  external  feedback  on  whether  they  fully  imderstood  each  function.  They  used 
their  own  judgment  on  when  to  proceed  to  the  final  exam. 
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Measures 


Pre-experiment  measures.  Before  starting  the  training,  soldiers  completed  a  survey  on 
their  computer  background  and  experience,  and  a  short  quiz  on  map  reading,  weapon  symbols, 
and  unit  symbols.  They  were  asked  whether  they  had  ever  used  the  Army’s  BR  numbering 
system.  Participants  also  answered  a  series  of  questions  directed  at  their  normal  tendency  to 
work  independently  -  to  solve  everyday  type  problems  vvdthout  requesting  help  or  assistance. 
Lastly,  they  were  asked  to  rate  the  extent  to  which  they  have  a  “knack”  for  learning  computer 
programs  on  their  own  and  a  tendency  to  figure  out  computer  shortcuts.  These  instruments  are  at 
Appendix  B. 

The  computer  survey  was  one  that  had  been  given  previously  to  over  2000  soldiers  as 
part  of  a  three-year  trend  analysis  of  the  computer  background  of  soldiers  (Dyer  &  Martin,  1999; 
Fober,  Bredthauer,  &  Dyer,  2000).  The  weapon  and  unit  symbols  tested  in  the  quiz  were 
identical  to  those  included  in  the  training,  except  that  the  mortar  symbol  was  used  instead  of  the 
SAW.  Thus  these  measures  constituted  a  test  of  their  current  knowledge  in  that  area.  The  OSUT 
soldiers  were  not  expected  to  know  these  symbols  as  that  is  not  a  subject  covered  in  initial 
training.  In  prior  research  (Centric,  Wampler,  &  Dyer,  1999),  it  was  found  that  military  symbols 
were  not  taught  in  BNCOC.  Consequently,  high  scores  from  the  BNCOC  soldiers  were  not 
expected  either.  The  map  reading  skills  tested  were  common  to  all  soldiers,  so  relatively  high 
scores  were  expected  on  these  items. 

The  questions  on  tendency  to  work  independently  were  developed  in  an  attempt  to 
identify  those  individuals  who  tend  to  figure  things  out  on  their  own  and  therefore  might  be  good 
in  an  exploratory  learning  environment.  Many  soldiers  take  this  independent  approach  first, 
rather  than  reading  instructions  or  asking  for  assistance.  A  search  of  the  literature  during  our 
planning  for  the  research  showed  no  prior  measures  of  this  tendency.  Therefore,  a  scale  was 
developed  in  which  participants  were  asked  how  they  would  usually  “approach”  each  of  fifteen 
activities  for  which  instructions  are  typically  available  (e.g.,  changing  the  settings  on  a  new 
digital  watch),  or  an  opportunity  to  ask  for  directions  (e.g.,  finding  your  seat  at  a  ballgame  or 
concert).  In  order  to  assess  whether  individuals  possessed  a  general  tendency  to  solve  problems 
on  their  own,  only  three  of  the  fifteen  items  described  computer-related  tasks.  The  remaining 
twelve  activities  were  quite  varied  (e.g.,  putting  together  a  piece  of  furniture,  responding  to  a 
survey).  The  scale  used  to  rate  the  15  tasks  was  as  follows: 

■  Complete  with  no  instructions:  Complete  the  activity  (or  give  up)  without  referring 
to  instructions,  a  map,  or  asking  for  directions. 

■  Read  instructions  only  if  needed:  Proceed  at  first  without  reading  instmctions,  a  map, 
or  asking  for  directions,  but  will  refer  to  these  upon  encountering  a  problem. 

■  Read  instructions  first:  Read  the  instructions,  a  map,  or  ask  for  help  before 
beginning. 

■  Find  someone  else  to  do  it:  Find  someone  else  to  perform  the  task  you  (a  friend, 
spouse,  etc.). 

■  Not  applicable 
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The  first  three  points  on  the  scale  were  felt  to  be  indicators  of  a  tendency  to  work 
independently,  that  is,  performing  the  task  without  instructions  or  guidance;  proceeding  at  first 
without  instructions,  but  referring  to  them  if  there  was  a  problem;  and  lastly  reading  instructions 
before  beginning  the  activity.  Soldiers  were  also  given  the  option  of  indicating  that  they  would 
find  someone  else  to  perform  a  task  for  them,  and  they  could  provide  a  rating  of  not  applicable 
for  tasks  that  they  had  never  performed.  The  tendency  to  work  independently  items  are  at 
Appendix  B. 

In  addition  to  assessing  tendencies  toward  working  independently,  the  survey  also  asked 
soldiers  to  provide  self-ratings  of  their  ability  to  navigate  through  a  computer  program  (computer 
knack)  and  their  ability  to  identify  efficient  methods  (short  cuts)  for  performing  computer  tasks. 
For  each  ability,  soldiers  rated  themselves  on  a  10-point  scale  ranging  from  “does  not  describe 
me  at  all”  (1)  to  “describes  me  completely”  (10). 

Criterion  measures.  Criterion  measures  were  scores  on  the  practical  exercises  embedded 
in  the  code  and  map  training,  final  exam  scores  on  the  code  and  map,  number  of  soldiers  who  did 
not  reach  the  80%  criterion  on  the  exercises,  and  time  required  to  complete  exercises  and 
training.  The  composition  of  these  exercises  and  exams  is  in  Tables  4  through  7. 

All  code  exercises  were  in  a  multiple-choice  format,  vidth  60  seconds  allowed  to  respond. 
Prior  to  each  coding  exercise  and  the  code  final  exam,  soldiers  were  shovra  the  type  of  item(s) 
they  would  encounter.  They  were  also  told  how  many  exercise  items  they  had  to  answer 
correctly,  corresponding  to  80%,  before  progressing  to  the  next  lesson.  If  they  did  not  achieve 
this  level,  they  went  through  a  review  and  retook  the  exercise.  The  reviews  were  specially 
developed  to  highlight  the  critical  parts  of  each  lesson.  They  were  not  simply  a  repetition  of  the 
lesson,  as  is  often  done  with  computer-based  training. 

With  the  first  weapon  and  unit  symbols  exercise,  correct/incorrect  feedback  was  provided 
immediately  after  answering  each  question.  When  soldiers  answered  incorrectly,  the  feedback 
also  provided  the  correct  response.  In  addition,  upon  completing  the  exercises,  soldiers  were 
informed  of  the  percentage  of  items  they  answered  correctly.  With  all  other  coding  exercises, 
feedback  was  provided  only  at  the  end  of  the  exercise.  In  this  case,  all  items  answered 
incorrectly  were  displayed,  and  the  correct  answer  was  shown.  Again,  soldiers  were  informed  of 
the  percentage  of  items  they  answered  correctly.  On  the  final  exam,  soldiers  were  only  informed 
of  the  percentage  of  questions  answered  correctly.  Examples  of  exercise  and  final  exam 
questions  on  the  coding  system  are  in  Appendix  D. 
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Table  4 

Description  of  Coding  Exercises 


Exercise  &  Condition 

Description  of  Exercise  Items 

Low  and  High  Demand  Conditions 

Weapon  &  Unit  Symbols 
(A) 

#  Items  =  24  (each  of  the  8  symbols  presented  3  times).  Response  was 
selected  from  a  multiple-choice  format  of  the  eight  symbol  names. 

BR  Numbering  System 
(B) 

#  Items  =  10.  Each  item  presented  two  BR  numbers.  Soldiers  had  to 
identify  the  echelon  (battalion,  company,  platoon,  squad,  individual)  at 
which  the  numbers  differed. 

Low  Demand  Condition 

Company  BR 
(C) 

#  Items  =  10.  Soldiers  were  shown  a  BR  number  and  had  to  identify  the 
correct  company,  from  a  list  of  the  five  companies:  Company  A  through 

D  and  HHC. 

Platoon  and  Squad  BR 
(D) 

#  Items  =  20  in  two  formats.  In  the  first  format,  soldiers  were  shown  a 

BR  number  and  asked  to  identify  the  squad  or  platoon.  In  the  second 
format,  a  verbal  description  of  an  individual  from  a  specific  unit  (e.g., 

“the  U'  Squad,  S'*  Platoon  of  Company  A”)  was  shown.  Soldiers  had  to 
select  the  correct  BR  number  from  five  choices. 

Rifle  Squad  Codes 
(E) 

#  Items  =  20.  Soldiers  were  shown  a  code  (weapon  symbol  plus  BR 
number).  They  had  to  identify  both  the  individual  and  his  squad.  In  the 
first  row  of  response  buttons,  the  three  squads  were  listed  (1st,  2d,  and  3d) 
in  random  order.  In  the  last  two  rows,  all  nine  duty  positions  in  the  rifle 
squad  were  listed,  in  random  order  (same  order  for  all  questions). 

Soldiers  had  to  answer  both  parts  correctly  to  “pass”  an  item. 

Weapons  Squad  Codes 
(F) 

#  Items  =15.  Soldiers  were  shown  the  code  of  a  soldier  in  a  weapons 
squad.  They  had  to  select  the  correct  duty  position  from  the  nine 
positions  in  the  weapons  squad  (presented  in  a  random  order). 

Key  Leader  Codes 
(G) 

#  Items  =  12  in  two  formats.  In  one  format,  the  leader  code  was  shown, 
and  the  soldier  had  to  select  the  description  of  the  leader’s  position  from 
five  options.  In  the  other  format,  the  leader’s  position  was  described,  and 
the  soldier  had  to  select  the  correct  code  from  five  options. 

Unit  Codes 
(H) 

#  Items  =  12  in  two  formats.  In  one  format,  the  unit  code  was  shown,  and 
the  soldier  had  to  select  the  correct  unit  description  from  one  of  five 
options.  In  the  other  format,  the  unit  was  described,  and  the  soldier  had  to 
select  the  correct  code  from  one  of  five  options. 

High  Demand  Condition 

Exercise  items  for  High  Demand  were  the  same  as  those  for  Low  Demand. 

Combined  Company 
through  Squad  BR  (C&D) 

#  Items  =  30. 

The  company,  platoon  and  squad  code  items  were  intermingled. 

Combined  Rifle  and 
Weapons  Squad  Codes 
(E&F) 

#  Items  =  35. 

The  rifle  squad  and  weapons  squad  items  were  intermingled. 

Combined  Key  Leader  and 

Unit  Codes 

(G&H) 

#  Items  =  24. 

The  key  leader  and  unit  items  were  intermingled. 
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The  final  exam  for  codes  is  described  in  Table  5.  As  noted  in  that  table,  a  single  item 
format  was  used  to  test  rifle  and  weapons  squad  members.  The  soldier  had  to  identify  the 
individual’s  squad  (1st,  2d,  3rd,  or  weapons)  as  well  as  the  individual  duty  position.  The  item 
was  scored  as  being  correct  only  if  the  soldier  answered  each  part  correctly.  A  percent  correct 
score”  was  given  upon  completing  the  code  final  exam.  There  was  no  feedback  on  individual 
test  items. 

Table  5 

Final  Exam  on  Codes 


Description  of  Code  Final  Exam  Items 

•  The  final  exam  on  codes  had  25  multiple-choice  items.  The  number  of  test  items  for  each 
code  was:  Leader  -  6;  unit  -  6;  rifle  squad  -  7;  weapon  squad  -  6.  Items  differed  from 
those  in  the  exercises. 

•  A  single  format  was  used  for  testing  rifle  and  weapon  squad  members.  The  code  (weapon 
and  BR)  for  an  individual  was  presented.  The  soldier  had  to  identify  both  the  individual’s 
squad  (1st,  2nd,  3rd  or  weapons)  and  his  duty  position  in  the  squad.  The  first  row  of 
response  buttons  displayed  the  squad  options.  The  last  three  rows  of  response  buttons 
displayed  the  individual  positions  (both  rifle  and  weapons  squad). 

•  For  all  unit  and  key  leader  items,  soldiers  were  presented  with  a  description  of  the  unit  or 
leader,  and  then  had  to  select  the  correct  code  from  a  total  of  5  options. 


For  most  map  exercises  (Table  6),  there  were  at  least  three  practical  applications  of  each 
function.  Soldiers  received  performance  feedback  immediately  after  each  response.  They  were 
also  informed  of  the  percentage  of  items  they  answered  correctly  at  the  end  of  each  exercise. 
Examples  of  map  lessons,  exercises,  and  final  exam  questions  are  in  Appendix  D. 
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Table  6 

Description  of  Map  Exercises 


Exercise  &  Condition 

Description  of  Map  Exercise  Items 

Low  Demand  Condition 

Zoom  In  and  Out,  Pan, 
and  Find  oneself 

(J) 

#  Items  =  12  (3  items/tasks  for  each  of  the  four  functions  listed  on  the  left). 
Soldiers  performed  tasks  that  used  each  of  the  four  functions.  For  each 
question,  soldiers  were  told  which  function  they  should  use.  For  example, 
an  item  on  the  “pan”  function  stated:  ""The  2“'  Platoon  Company  C  is  north 
and  west  of  you.  Pan  the  map  display  until  the  unit  symbol  is  visible  near 
the  center  of  the  map  display.  Once  the  platoon  symbol  is  centered  on  your 
map  display,  click  on  the  symbol." 

Soldiers  were  scored  on  whether  they  used  the  required  function.  If  asked 
to  click  on  a  symbol,  they  were  not  penalized  if  they  failed  to  do  so.  The 
immediate  feedback  told  them  whether  they  had  correctly  used  the  required 
function,  as  well  as  whether  they  had  clicked  the  correct  map  symbol. 

FindX 

(K) 

#  Items  =  12. 

Soldiers  had  to  use  the  “Find  X”  function  to  locate  individuals  or  units. 

For  example:  “  Using  FindX,  find  ISG  Company  C.  When  you  find  the 
corresponding  symbol  on  the  map,  click  on  it." 

To  receive  credit  in  this  exercise,  soldiers  had  to  click  on  the  correct  map 
symbol  once  it  was  located.  If  they  found  the  wrong  individual/unit  or 
failed  to  click  on  the  symbol,  they  were  told  that  they  had  not  completed 
the  task  successfully. 

Display  Others 
(L) 

#  Items  =16  (only  8  counted  toward  a  score). 

The  8  tasks  that  were  scored  required  soldiers  to  use  the  “Display”  function 
to  display  groups  of  individuals  or  units.  For  example:  ""You  are  the 
commander.  Company  A.  Display  all  platoon  leaders.  Click  on  the  3  Pit 
Ldr."  Soldiers  were  not  penalized  if  they  failed  to  click  on  the  correct 
symbol,  as  long  as  they  displayed  the  correct  group  of  individuals  or  units. 

The  questions  that  were  not  scored  asked  soldiers  to  click  on  a  particular 
unit  or  individual  already  displayed  on  the  map.  For  example,  ""You  are 
still  the  commander.  Company  A.  Click  on  the  2f  Platoon  Leader ." 

Soldiers  did  receive  immediate  feedback  for  these  items. 

High  Demand  Condition 

Exercise  items  were  the  same  as  those  in  the  Low  Demand  condition. 

Zoom  In/Out,  Pan,  Find 
oneself.  Find  X  and 
display  others 
(J-L) 

Soldiers  were  exposed  to  all  map  exercises  at  a  single  point  in  time.  The 
exercises  were  in  the  same  order  as  in  the  Low  condition.  The  items  were 
not  intermingled  as  was  done  with  the  High  Demand  Code  condition. 
Consequently,  soldiers  received  a  score  for  each  of  the  three  separate 
exercises,  not  a  single  score  across  all  exercise  items. 

Note.  There  were  no  practice  exercises  for  Lesson  M  on  determining  range  and  azimuth.  It  was 
determined  that  no  exercises  were  required,  as  the  words  associated  with  the  range/azimuth  button  were 
almost  self-explanatory. 
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The  map  final  exam  (Table  7)  covered  all  seven  fimctions/buttons.  The  questions  also 
required  that  the  soldiers  identify  units  and  individuals;  the  material  covered  in  Phase  I  of  the 
experiment.  Feedback,  percent  correct,  was  provided  upon  completion  of  the  exam.  There  was 
no  feedback  on  individual  test  items. 

Table  7 

Final  Exam  on  Map  Functions 


•  The  map  final  exam  had  22  items,  with  some  two-part  questions,  for  a  total  of  30  points. 

Items  worth  2  points  required  soldiers  to  perform  two  functions  to  complete  the  task.  Partial 
credit  was  given  when  one  of  the  two  functions  was  performed  correctly  on  a  2-point  item. 
Test  items  differed  from  those  in  the  exercises. 

•  Of  the  22  items,  3  directly  tested  knowledge  of  the  BR  numbering  system  by  asking  soldiers 
to  view  the  codes  for  nine  squad  members  and  identify  which,  if  any,  were  from  a  different 
squad. 

•  The  remaining  items  asked  soldiers  to  accomplish  a  variety  of  tasks  such  as  locating  points 
on  the  map,  displaying  and  locating  individuals  and  units,  and  determining  range  and  azimuth 
between  two  points.  With  the  exception  of  the  range/azimuth  tasks,  there  was  typically  more 
than  one  way  to  accomplish  a  task,  although  there  was  clearly  a  most  efficient  function  for 
each  task.  Therefore,  soldiers  were  not  graded  on  the  method  used.  Instead,  each  task 
required  them  to  click  on  a  map  symbol  or  location,  or  to  type  a  response  in  a  text  box  to 
indicate  their  successful  accomplishment  of  the  task.  The  one-minute  time  limit  per  question 
made  it  difficult  for  soldiers  to  respond  successfully  without  using  the  most  efficient  function 
for  task  completion. 

•  This  was  the  first  time  soldiers  were  required  to  execute  range/azimuth  tasks,  although  those 
in  the  Exploratory  condition  had  the  opportunity  to  work  with  this  function. 
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Results 


Pre-experiment  Measures 

Computer  background.  In  general,  the  computer  backgrounds  of  the  soldiers  (see  survey 
in  Appendix  B)  were  very  similar  to  that  obtained  in  prior  research  (Dyer  &  Martin,  1999;  Fober 
et  al.,  2000).  The  major  deviation  from  these  previous  findings  was  that  BNCOC  soldiers  in  the 
experiment  were  lower  on  subjective  and  objective  measures  of  computer  expertise  than  the 
OSUT  soldiers.  In  the  prior  research,  OSUT  soldiers  were  lower  than  BNCOC. 

Table  8  shows  the  percentage  of  soldiers  who  used  a  computer  at  some  time  in  their 
formal  schooling,  the  percentage  currently  using  a  computer,  and  the  percentage  that  own  a 
computer  (see  also  Appendix  C).  The  lack  of  use  of  computers  in  public  school  settings  directly 
reflects  the  ages  of  the  soldiers  in  the  different  groups.  The  youngest  groups  (OSUT  and  lOBC) 
were  more  likely  to  have  been  exposed  to  computers  at  the  grade  school,  junior  high,  and  high 
school  levels  (see  Table  C-4).  On  the  other  hand,  the  ANCOC  soldiers  were  the  least  likely  to 
have  been  exposed  to  computers  even  during  high  school.  The  picture  changed  somewhat  with 
college  experience  as  most  colleges  require  students  to  use  computers.  The  lOBC  soldiers  were 
the  mostly  likely  to  have  used  computers  in  college  (95%),  followed  by  ANCOC  (52%),  then 
BNCOC  (40%)  and  lastly  OSUT  (23%). 

Table  8 

Use  of  Computers  by  Soldier  Group 


Soldier  Group 

%  Used  Computer 
During  Their  Formal 
Education 

%  Currently  Using  a 
Computer 

%  Owning  a 
Computer 

OSUT 

96% 

63% 

58% 

BNCOC 

81% 

98% 

67% 

ANCOC 

76% 

97% 

90% 

lOBC 

100% 

95% 

93% 

The  percentage  of  soldiers  within  each  group  currently  using  computers  was  very  high,  at 
least  95%  for  all  groups  but  the  OSUT  soldiers  (63%).  For  each  group,  the  usage  percentages 
tended  to  be  slightly  higher  than  the  percentage  of  soldiers  indicating  they  owned  a  computer. 

As  shovm  in  Table  8,  the  ownership  rates  were  highest  for  lOBC  and  ANCOC,  lower  for 
BNCOC,  and  lowest  for  OSUT.  Of  interest  is  that  over  half  the  OSUT  soldiers  said  they  owned 
a  computer.  Both  the  computer  ownership  and  computer  use  variables  showed  significant  soldier 
group  differences  (see  Table  C-6). 

With  regard  to  all  computer  features  used  (mouse,  games,  internet,  etc.),  there  were  no 
significant  differences  among  the  groups  (see  Table  C-7  in  Appendix  C).  The  groups  did  differ 
on  their  self-ratings  of  typing  skill,  with  the  lOBC  soldiers  most  likely  to  say  they  could  type 
quickly  (45%)  and  BNCOC  the  least  likely  to  have  this  skill  (8%),  y^{9)  =  27.83,/?=  .0010. 
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Both  subjective  and  objective  indices  of  computer  skill  were  derived  from  the  survey. 
The  subjective  index  was  the  soldier’s  own  rating  of  his  skill  on  a  six-point  scale  from  “novice” 
to  “Bill  Gates  would  hire  me.”  The  other  index  was  based  on  a  quiz  that  required  soldiers  to 
identify  1 8  icons  commonly  used  in  Windows-based  software.  Significant  differences  were 
found  among  the  groups  on  both  indices;  self-rating,  F(3,  164)  =  7.38,/?  =  .0001;  icon  quiz,  F(3, 
164)  =  7.89,/?  -  .0001 .  For  both  indices,  the  lOBC  soldiers  were  higher  than  soldiers  in  each  of 
the  other  courses.  In  addition,  on  the  icon  quiz,  lOBC  scores  were  higher  than  BNCOC  scores. 
Table  9  presents  summary  descriptive  statistics  on  each  index.  Additional  information  is  in 
Appendix  C. 

Table  9 

Means  on  Self-ratings  of  Computer  Skill  and  on  Icon  Scores 


Soldier 

Group 

N 

Self  Rating 

Icon  Score 

M 

SD 

M 

SD 

OSUT 

48 

2.04 

0.97 

8.92 

3.38 

BNCOC 

48 

1.89 

0.91 

7.97 

3.49 

ANCOC 

30 

2.00 

0.91 

9.53 

3,12 

lOBC 

43 

2.76 

1.01 

11.17 

2.58 

At  the  group  level,  there  was  agreement  between  the  objective  and  subjective  of  indices 
of  computer  skill.  From  high  to  low,  the  group  order  was  similar:  lOBC,  ANCOC  and  OSUT, 
and  then  BNCOC.  ‘  To  illustrate  the  relationship  between  the  objective  and  subjective  indices  at 
the  group  level,  consider  the  lOBC  group,  which  was  highest  on  both  indices,  and  the  BNCOC 
group,  which  was  lowest  on  both  indices.  For  lOBC  soldiers  20%  indicated  they  could  program 
in  a  language  such  as  C++  or  Visual  Basic,  and  only  12%  rated  themselves  as  computer  novices. 
In  addition,  62%  of  the  lOBC  soldiers  correctly  answered  over  half  the  items  on  the  icon  test. 

On  the  other  hand,  44%  of  soldiers  in  BNCOC  rated  themselves  as  novices  and  only  2% 
indicated  they  could  program.  Only  33%  of  the  BNCOC  soldiers  correctly  answered  over  half 
the  items  on  the  icon  test. 

As  a  summary.  Figure  6  graphically  illustrates  how  the  soldier  groups  ordered  on  four 
major  variables  from  the  computer  survey:  percent  soldiers  using  a  computer,  percent  soldiers 
owning  a  computer,  percent  soldiers  indicating  they  were  not  computer  novices,  and  the  mean 
percent  correct  on  the  icon  test.  lOBC  soldiers  were  the  highest  on  each  dimension.  Of  interest 
is  that  even  though  OSUT  soldiers  had  the  lowest  usage  and  ownership  rates,  they  were  similar 
to  BNCOC  and  ANCOC  with  respect  to  self-ratings  and  icon  scores.  The  graph  also  illustrates 
that  high  rates  of  soldiers  owning  and  using  a  computer  within  a  group  did  not  necessmily 
translate  to  high  self-ratings  for  the  group  as  a  whole. 


'  This  statement  should  not  be  interpreted  to  mean  that  each  group  differed  significantly  from  each  other  group  on 
each  of  these  indices. 
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Figure  6,  Summary  of  computer  usage  and  ownership,  and  subjective  and  objective  indices  of 
computer  expertise. 

Military  Knowledge:  Military  symbols^  the  battle  roster  system^  and  map  reading. 
OSUT  soldiers  were  expected  to  score  lowest  on  these  initial  tests,  and  ANCOC  soldiers  were 
expected  to  perform  better  than  BNCOC  soldiers.  It  was  expected  that  lOBC  soldiers  would 
score  as  high  as  ANCOC  and  in  some  cases,  higher.  In  general,  the  test  results  were  consistent 
with  these  expectations. 

The  mean  percent  correct  on  each  test  for  each  soldier  group  is  in  Table  10.  Clearly,  each 
group  of  soldiers  was  more  knowledgeable  about  map  reading  than  symbols,  although  ANCOC 
soldiers  performed  well  on  the  unit  symbol  test. 

Table  10 

Mean  Percent  Correct  on  Military  Knowledge  Tests 


Test 

Soldier  Group 

All  Groups 

OSUT 

BNCOC 

ANCOC 

lOBC 

Symbols 

Weapon 

2% 

12% 

15% 

29% 

14% 

Unit 

1% 

15% 

73% 

54% 

31% 

Map  Reading 

55% 

86% 

85%  1 

78% 

75% 
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As  indicated  in  Table  10,  most  soldiers  did  not  know  the  individual  weapon  symbols. 

The  highest  weapon  score  was  for  the  lOBC  soldiers;  45%  of  them  correctly  identified  the 
symbol  for  the  Ml 6  rifle.  Many  soldiers  did  not  even  attempt  to  answer  these  items  (see  Table 
C-13).  The  non-response  percentages  were  very  high  for  the  OSUT  soldiers  (at  least  96%).  The 
percentage  of  BNCOC  soldiers  who  did  not  respond  to  these  items  was  lower,  but  also  high, 
ranging  from  44%  to  71%.  Even  within  ANCOC  and  lOBC,  some  weapon  symbols  elicited  a 
high  non-response  rate.  Groups  differed  significantly  on  weapons  (see  Table  C-14).  lOBC 
soldiers  had  higher  scores  than  each  of  the  other  groups,  and  BNCOC  and  ANCOC  were  higher 
than  OSUT.  Overall,  soldiers  were  most  likely  to  know  the  symbol  for  the  M16  rifle  (19%),  and 
least  likely  to  know  the  symbol  for  the  M203  grenade  launcher  (9%). 

With  regard  to  the  imit  symbols,  ANCOC  and  lOBC  soldiers  exhibited  more  knowledge 
of  these  symbols  than  weapon  symbols.  The  unit  symbol  scores  for  OSUT  and  BNCOC  were 
low  and  similar  to  their  weapon  symbol  scores.  As  might  be  expected,  there  was  an  inverse 
relationship  for  the  groups  between  non-response  rates  and  unit  symbol  scores  (see  Table  C-15). 
For  example,  OSUT  soldiers  had  a  very  high  non-response  rate  (85%)  and  averaged  less  than  2% 
correct  on  unit  symbols.  Again,  there  were  significant  differences  between  groups  (see  Table  C- 
16).  The  order  from  high  to  low  was  ANCOC,  lOBC,  BNCOC,  and  lastly  OSUT. 

Map  reading  scores  were  generally  high  (see  Table  10).  Although  OSUT  soldiers  scored 
the  lowest,  their  average  score  was  55%  correct.  lOBC,  ANCOC,  and  BNCOC  soldiers  each 
scored  significantly  higher  than  OSUT  soldiers  (see  Tables  C-17  and  C-18). 

With  regard  to  the  BR  system,  only  the  noncommissioned  officers  indicated  either  being 
trained  on  or  having  used  this  system.  These  percentages  were  lower  for  BNCOC  than  ANCOC 
soldiers,  with  21%  of  ANCOC  soldiers  saying  they  had  been  trained  and  38%  stating  they  had 
used  the  system  (see  Table  C-19).  None  of  the  OSUT  soldiers  had  any  experience  with  this 
system,  and  less  than  8%  of  the  lOBC  soldiers  indicated  experience.  No  statistical  tests  of 
significance  were  computed  on  BR  data  because  of  the  disproportionate  numbers  of  soldiers  and 
OSUT  and  lOBC  without  BR  experience. 

In  summary,  the  soldiers  entered  the  experiment  with  different  levels  of  military 
knowledge.  All  groups  showed  good  map  reading  skills,  with  OSUT  the  lowest  in  this  area. 

Most  ANCOC  soldiers  and  about  half  the  lOBC  soldiers  knew  the  imit  symbols,  but  OSUT  and 
BNCOC  soldiers  did  not  know  these  symbols.  None  of  the  groups  knew  all  the  weapon  symbols. 
About  one-third  of  the  ANCOC  soldiers  had  had  experienee  or  training  with  the  battle  roster 
system,  with  the  experience  of  the  other  soldiers  much  less  or  non-existent. 

Tendency  to  work  independently.  Responses  to  the  fifteen  tasks  on  the  tendency  to  work 
independently  scale  were  examined  to  determine  if  any  items  should  be  deleted  as  not 
representing  “typical”  tasks.  A  breakdown  of  responses  to  each  item  is  in  Table  C-20.  Two 
items  were  eliminated  from  further  analyses  because  of  atypical  response  patterns.  Responses  to 
the  item  “take  a  new,  over-the-counter  medication”  did  not  vary.  More  than  four-fifths  (83%)  of 
the  soldiers  indicated  they  would  read  the  instructions  before  taking  a  new  medication.  Another 
item,  “fill  out  tax  forms,”  generated  a  unique  distribution  of  responses.  One-third  (33%) 
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reported  they  would  find  someone  else  to  perform  this  task  for  them,  while  less  than  10% 
responded  this  way  to  any  of  the  other  tasks.  Further  analyses  were  conducted  on  the  remaining 
13  items. 

To  examine  whether  soldier  groups  differed  in  how  they  responded  to  the  remaining  13 
items,  five  new  variables  were  created.  For  each  soldier,  the  number  of  tasks  performed  in 
accordance  with  each  response  option  was  calculated.  In  other  words,  we  tallied  the  number  of 
tasks  completed  with  no  instructions,  the  number  where  instructions  were  read  only  if  needed, 
the  number  where  individuals  found  someone  else  to  do  it,  and  the  not  applicable  responses. 
Since  thirteen  tasks  were  analyzed,  possible  scores  for  each  response  ranged  from  zero  to 
thirteen.  Mean  scores  for  each  approach  are  shown  in  Table  C-21  by  soldier  group. 

The  tendency  to  work  independently  is  an  individual  difference  variable  that  should  not 
relate  to  soldier  group.  Nor  should  this  construct  relate  to  the  experimental  map  training 
conditions.  To  test  for  possible  differences,  a  soldier  group  by  map  condition  ANOVA  was 
performed  for  each  of  the  five  task  approaches.  There  were  no  significant  differences  in  mean 
scores  for  each  approach  by  soldier  group  or  map  condition  (Table  C-22). 

On  average,  the  soldiers  indicated  they  would  read  instructions  for  41%  of  the  thirteen 
tasks  (M=  5.36,  SD  =  3.08).  They  would  put  off  reading  instructions  until  a  problem  arose  for 
36%  of  the  tasks  (M  =  4.69,  SD  =  2.79).  Soldiers  only  mentioned  two  to  three  tasks  (19%)  that 
they  would  perform  without  referring  to  instructions  (M=  2.45,  SD  =  2.79). 

A  k-means  cluster  analysis  was  conducted  to  determine  the  percentage  of  soldiers  who 
would  fall  into  what  could  be  considered  an  independent  style  of  solving  problems  versus  a  more 
dependent  style.  The  “not  applicable”  and  “find  someone  else  to  do  it”  responses  were  excluded 
from  the  analysis.  Two  clusters  of  soldiers  emerged.  The  typical  response  by  soldiers  in  each 
cluster  to  the  tasks  in  the  scale  was  calculated.  The  results  are  shovm  in  Figure  7.  The  figure 
also  shows  that  fewer  soldiers  were  classified  as  solving  tasks  independently  than  in  a  more 
dependent  manner  (29%  vs.  71%). 


N  =  48  Cluster  N  =  120 

Figure  7.  Profile  of  task  completion  strategies  by  soldier  tendency  to  work  independently. 
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Code  Training  Results 


With  the  code  training,  the  primary  interest  was  in  determining  whether  there  were 
differences  between  the  High  and  Low  Demand  conditions  and  among  the  soldier  groups,  as  well 
as  any  interactions  between  these  two  factors.  The  criterion  variables  were  scores  on  the 
exercises  and  final  exam,  time  to  complete  the  training,  and  number  of  soldiers  who  had  to  repeat 
an  exercise  because  of  failure  to  score  at  least  80%. 

Comparisons  were  made  on  the  following  five  exercise  scores  plus  the  final  exam. 

■  Weapon  and  unit  symbols 

■  Battle  roster  system 

■  Company,  platoon  and  squad  codes  (for  the  Low  condition,  this  score  combined 
the  company  exercise  and  the  platoon/squad  exercise) 

■  Rifle  and  weapon  squad  codes  (for  the  Low  condition,  this  score  combined  the 
rifle  squad  exercise  and  the  weapons  squad  exercise) 

■  Key  leader  and  unit  codes  (for  the  Low  condition,  this  score  combined  the  key 
leader  exercise  and  the  unit  code  exercise) 

Because  the  first  two  blocks  of  instruction  (weapon  and  unit  symbols,  the  BR  system)  were 
identical  for  the  High  and  Low  Demand  conditions,  no  differences  were  expected  on  these.  In 
addition,  because  soldiers  in  each  condition  had  to  repeat  an  exercise  if  they  scored  below  80%, 
differences  between  High  and  Low  Demand  conditions  were  not  expected  on  the  final  exam. 
Consequently,  there  were  only  three  variables  for  which  differences  in  the  High  and  Low 
conditions  were  likely  to  occur:  Company-squad  codes,  rifle  and  weapon  squad  members;  key 
leader  and  unit  codes.  All  analyses,  unless  stated  otherwise,  are  based  on  the  first  time  soldiers 
took  the  exercise. 

Scores  on  code  exercises  and  code  final  exam.  A  two-way  analysis  of  variance  (training 
condition  by  soldier  group)  was  conducted  on  each  exercise  score  and  the  final  exam  score  (see 
Table  E-1,  Appendix  E).  There  were  main  effects  for  both  factors,  but  no  significant 
interactions.  There  was  one  significant  effect  for  the  Low-High  condition,  and  three  significant 
effects  for  soldier  group. 

The  only  score  with  a  difference  between  the  High  and  Low  Demand  conditions  was  on 
the  exercises  involving  the  rifle  and  weapon  squads,  i^(l,  159)  =  23.53, p  =  .0000.  In  this  case, 
the  soldiers  in  the  Low  Demand  group  scored  better  than  those  in  the  High  Demand  group. 

Figure  8  shows  the  means  for  both  groups  on  the  five  exercise  scores  and  the  final  exam.  The 
graph  shows  clearly  the  impact  of  the  High  Demand  condition  on  rifle  and  weapons  squad  code 
scores.  Descriptive  statistics  for  each  condition  are  in  Table  E-2. 
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Code  Exercises  and  Final  Exam 

Figure  8.  Mean  scores  on  the  code  exercises  and  code  final  exam  for  the  High  and  Low  Demand 
conditions.  (Scores  with  significant  differences  between  High  and  Low  are  highlighted.) 

On  the  rifle  and  weapon  squads  score,  it  is  important  to  remember  that  the  Low  Demand 
score  represents  the  combined  score  from  two  separate  exercises,  as  the  instruction  for  the  Low 
condition  was  divided  into  two  blocks  of  instruction  (rifle  and  weapon  squads  codes).  This  part 
of  the  instruction  also  involved  the  greatest  differences  in  working  demand  for  the  two 
conditions  (an  estimated  9  chunks  per  lesson  versus  1 8  chunks  per  lesson,  reference  Table  2), 
and  the  greatest  potential  for  confusion.  Soldiers  had  to  learn  two  squad  sets  of  positions,  each 
associated  with  numbers  0  through  8.  In  both  cases,  the  squad  leader  was  coded  a  0,  but,  as 
indicated  below,  this  was  the  only  commonality  across  the  number  assignments  in  the  two  types 
of  squads. 


Number  Rifle  Squad  Position 
0  Squad  Leader 

1  A  Team  Leader 

2  SAW  Gunner,  A  Tm 

3  Grenadier,  A  Tm 

4  Rifleman,  A  Tm 

5  B  Team  Leader 

6  SAW  Gunner,  B  Tm 

7  Grenadier,  B  Tm 

8  Rifleman,  B  Tm 


Weapons  Squad  Position 

Squad  Leader 

1st  Machine  Gunner 

Assistant  Gunner  to  1st  Machine  Gurmer 
2d  Machine  Gunner 

Assistant  Gunner  to  2d  Machine  Gunner 
1st  Javelin  Gunner 

Assistant  Gunner  to  1  st  Javelin  Gunner 

2nd  Javelin  Gunner 

Assistant  Gunner  to  2d  Javelin  Gunner 
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The  weapon  symbol  associated  with  each  duty  position  helped  to  distinguish  some 
positions.  However,  the  rifle  weapon  symbol  was  common  to  five  soldiers  in  both  the  rifle  and 
weapons  squads. 

An  item  analysis  was  conducted  on  the  rifle  and  weapons  squad  exercises  to  ascertain 
whether  soldiers  in  the  High  Demand  condition  had  difficulty  with  certain  items  or  if  the  lower 
performance  was  spread  across  questions.  The  results  indicated  the  lower  exercise  scores 
reflected  both  factors.  Questions  on  rifle  squad  member  codes  had  two  parts.  The  stem  for  all 
items  was  a  weapon  symbol/BR  code,  as  shown  in  Figure  9.  The  soldiers  had  to  identify  the  rifle 
squad  (1  st,  2d,  or  3d)  as  well  as  the  individual  position  within  the  squad.  The  first  three  such 
items  in  the  exercise  created  more  difficulties  for  soldiers  in  the  High  than  the  Low  Demand 
condition.  In  particular,  those  in  the  High  condition  tended  to  identify  only  the  squad  member 
himself,  not  the  squad.  The  item  difficulties  or  percent  correct  responses  for  these  first  three 
questions  were  .48,  .72,  and  .78  for  High  Demand  versus  .80,  .92,  and  .84  for  Low  Demand. 

This  level  of  difficulty  for  High  Demand  did  not  occur  again;  the  item  difficulties  showed  a 
learning  effect  (a  recognition  by  soldiers  that  they  had  to  identify  the  squad  as  well).  It  is 
important  to  note  that  the  High  Demand  condition’s  exercise  involved  both  rifle  squad  and 
weapons  squad  questions.  In  contrast,  the  Low  Demand  condition  had  two  separate  exercises, 
one  on  rifle  squad  and  one  on  weapons  squad. 


2.  Identify  both  the  individual  squad  member  and 
his  squad. 


Figure  9.  Example  of  an  exercise  item  on  squad  member  codes. 

When  the  item  difficulties  for  the  remaining  rifle  squad  items  were  examined,  the  mean 
difficulty  was  .83  for  High  Demand  and  .93  for  Low  Demand,  indicating  a  lower  level  of 
performance  for  the  High  Demand  condition.  A  similar  pattern  occurred  on  the  weapons  squad 
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items.  Here  the  mean  item  difficulty  was  .82  for  High  Demand  and  .93  for  Low  Demand.  It 
appeared  that  the  significantly  lower  scores  for  the  High  Demand  condition  resulted  from 
soldiers  encountering  different  item  formats  within  an  exercise  as  well  as  a  general  lower  level  of 
performance,  presumably  from  the  higher  demands  placed  on  working  memory.  Lastly,  it  is 
important  to  stress  that  all  exercises  were  preceded  by  a  presentation  of  the  type  of  items  the 
soldiers  would  encounter  during  the  exercises,  so  soldiers  would  understand  what  was  required 
of  them. 

Significant  differences  occurred  among  the  soldier  groups  on  two  of  the  five  code 
exercise  scores  and  the  code  final  exam,  F(3, 159)  =  6A9,p  —  .0003.  The  two  exercise  scores 
with  significant  soldier  group  differences  were  the  rifle  and  weapons  squad  score  [F(3, 159)  = 
6.26. p  =  .0004]  and  the  key  leader  and  unit  score  [F(3,  159)  =  5.51,/?  =  .0012].  For  each,  lOBC, 
ANCOC,  and  BNCOC  soldiers  scored  higher  than  OSUT  soldiers  (see  Table  E-3).  On  the  final 
exam,  lOBC  also  scored  higher  than  BNCOC. 

The  means  for  the  four  soldier  groups  on  all  code  scores  are  in  Figure  10.  The  graph 
shows  the  lack  of  significant  differences  among  the  groups  on  the  first  part  of  the  instruction. 
Beyond  that  point,  the  lower  performance  of  the  OSUT  soldiers  is  demonstrated,  as  is  the  lower 
performance  by  BNCOC  on  the  final  exam.  Of  additional  interest  is  the  general  ordering  of  the 
soldier  groups  from  high  to  low  as  lOBC,  ANCOC,  BNCOC,  and  then  OSUT. 
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Figure  10,  Mean  scores  on  the  coding  exercises  and  code  final  exam  for  each  soldier  group. 
(Scores  with  significant  soldier  group  differences  are  highlighted.) 
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It  might  be  expected  that  the  number  of  soldiers  who  had  to  review  a  lesson  and  repeat 
the  corresponding  exercise  would  be  reflective  of  the  impact  of  working  memory  demands  on 
performance.  The  number  of  soldiers  who  had  to  repeat  each  block  of  instruction  is  in  Table  1 1 . 

Table  11 

Number  of  Soldiers  who  had  to  Repeat  Code  Exercises 


Block  of  Instruction/Exercises 

Low  Demand  («  =  84) 

High  Demand  {n  =  83) 

Exercises  that  were  the  same  in  both  conditions 

1  (1%) 

3  (4%) 

3  (4%) 

4  (5%) 

Exercises  that  varied  with  condition 
(Information  in  smaller  chunks  for  Low  Demand) 

Company  BR 

0  (0%) 

4  (5%) 

Platoon  -  Squad  BR 

0  (0%) 

Rifle  Squad  Codes 

2  (2%) 

13  (16%) 

Weapons  Squad  Codes 

4  (5%) 

Key  Leader  Codes 

12  (14%) 

11  (13%) 

Unit  Codes 

17(20%) 

Note.  Number  of  soldiers  who  had  to  repeat  exercises  within  each  course  regardless  of  training 
condition:  34  in  OSUT;  20  in  BNCOC;  16  in  ANCOC;  6  in  lOBC. 


With  respect  to  remediation,  as  expected,  the  first  two  blocks  of  instruction  showed  no 
difference  between  the  two  conditions,  as  the  amount  of  information  presented  was  the  same 
under  both  conditions.  With  the  next  blocks  of  instruction  on  company  through  squad  BR,  no 
soldiers  had  to  repeat  the  exercise  in  the  Low  Demand  group,  as  compared  to  5%  in  the  High 
Demand  group.  With  the  rifle/weapons  squad  instruction,  twice  as  many  in  the  High  Demand 
group  had  to  repeat  the  exercise.  These  percentages  were  consistent  with  the  expectation  that  the 
High  Demand  condition  would  not  enable  soldiers  to  acquire  the  information  as  easily  as  the 
Low  Demand  condition.  However,  this  pattern  was  not  repeated  on  the  last  blocks  of  instruction 
on  key  leader  and  unit  codes.  In  fact  the  percentages  of  soldiers  in  the  Low  Demand  condition 
who  had  to  repeat  key  leader  and  unit  codes  increased.  They  were  14%  and  20%  respectively; 
much  different  from  the  0%  to  5%  remediation  rates  for  Low  Demand  in  the  other  exercises. 

An  item  analysis  was  conducted  to  try  to  determine  what  caused  this  increase.  The 
analysis  showed  that  the  exercise  question  format  in  conjunction  with  the  total  number  of 
questions  in  each  exercise  affected  the  results,  even  though  the  overall  item  difficulty  was  the 
same  for  both  conditions  (.89  probability  of  being  correct  in  both  cases).  An  unexpected  item 
format  may  have  “caught  soldiers  off  guard.” 

Exercises  for  the  key  leader  and  unit  codes  had  two  types  of  questions.  These  two 
formats  are  illustrated  in  Figure  1 1  for  the  unit  questions.  In  the  format  shown  at  the  top  of 
Figure  1 1,  the  item  stem  was  the  unit  symbol/BR  number  combination  with  the  multiple-choice 
alternatives  being  word  descriptions  of  possible  units.  Of  the  12  questions,  7  were  this  format. 
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In  the  other  format,  shown  at  the  bottom  of  Figure  1 1 ,  the  item  stem  was  a  verbal  description  of  a 
unit;  the  alternatives  were  different  unit  symbol/BR  combinations.  There  were  5  questions  in 
this  format.  Three  had  options  where  the  codes  had  the  identical  BR  number  associated  with 
different  unit  symbols.  These  three  questions  created  problems  for  all  soldiers  (answered 
correctly  by  70%  or  less  within  each  condition).  The  item  analysis  showed  the  soldiers  had  the 
BR  number  correct,  but  did  not  discriminate  between  the  imit  symbols.  For  example,  in  Figure 
1 1  the  correct  BR  is  AB23 1  for  B  Co,  3d  Platoon,  1st  Squad.  However,  the  unit  symbol  options 
included  squad,  platoon,  and  company  symbols.  Soldiers  often  incorrectly  checked  the  code 
with  the  company  or  platoon  symbol,  vice  squad  symbols,  that  also  had  the  correct  BR  number. 


Correct  >-| 

1 - 1  Incorrect 

Figure  11.  Illustrations  of  the  two  question  formats  in  the  key  leader  and  unit  code  exercises. 


The  single  exercise  in  the  High  Demand  condition  had  twice  as  many  items  as  each 
exercise  in  the  two  Low  Demand  conditions.  As  a  result,  two  soldiers’  overall  performance 
could  have  been  the  same  in  the  High  and  Low  Demand  conditions,  but  one  would  have  passed 
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the  exercise  the  first  time  and  the  other  failed  it.  For  example,  if  a  soldier  missed  all  three  unit 
items  in  Low  Demand  (3  of  a  total  of  12  questions,  75%  correct),  he  would  have  had  to  repeat 
the  exercise.  If  a  soldier  missed  all  three  items  in  High  Demand  (3  of  a  total  of  24  questions, 

87%  correct),  he  would  have  passed. 

An  analysis  of  the  leader  items  was  also  conducted.  But  there  was  no  distinct  pattern 
associated  with  the  item  stem  as  occurred  with  the  unit  questions.  There  is  no  readily  available 
explanation  for  the  relatively  high  percentage  of  soldiers  who  had  to  repeat  the  leader  questions 
within  the  Low  Demand  condition. 

Considering  all  conditions  and  exercises,  OSUT  soldiers  were  most  likely  to  repeat 
exercises,  followed  by  BNCOC  and  ANCOC,  and  lastly  lOBC.  About  half  the  OSUT  soldiers 
(48%)  repeated  at  least  one  exercise,  compared  to  33%  of  the  BNCOC  and  ANCOC  soldiers  and 
to  12%  of  the  lOBC  soldiers.  This  represented  a  significant  difference  among  the  courses,  x^(3) 
=  13.07,  p  =  .0044.  Of  all  the  soldiers  who  had  to  repeat  an  exercise(s),  in  two-thirds  of  the  cases 
they  only  had  to  repeat  one.  In  addition,  for  those  who  had  to  repeat  exercises,  OSUT  soldiers 
were  most  likely  to  repeat  more  than  one  exercise,  while  lOBC  soldiers  were  least  likely  to  do  so 
(19%  vs.  2%  respectively). 

Times  on  code  exercises  and  code  final  exam.  Time  to  complete  each  code  block  of 
instruction  (lessons  and  exercises)  and  the  final  exam  was  also  examined.  A  two-way  analysis  of 
variance  (training  condition  by  soldier  group)  was  conducted  on  time  (see  Table  E-4).  The  time 
data  included  the  time  spent  repeating  exercises  for  those  soldiers  who  did  not  meet  the  80% 
criterion. 

There  were  main  effects,  but  no  significant  interactions  (Table  E-4).  The  only  significant 
difference  between  the  High  and  Low  Demand  conditions  on  time  was  on  the  rifle  and  weapons 
squad  member  blocks  of  instruction.  Soldiers  in  Low  Demand  completed  the  same  instruction  in 
less  time  than  those  in  High  Demand,  FXL  195)  =  16.32,/?  =  .0001  (Table  E-5). 

For  the  soldier  group,  there  was  a  significant  effect  on  four  blocks  of  instruction  (weapon 
and  unit  symbols;  company-squad  codes,  rifle-weapons  squad  member  codes,  and  key  leader  and 
unit  codes).  Descriptive  statistics  are  in  Table  E-6.  lOBC  and  ANCOC  were  significantly  faster 
than  OSUT  and  BNCOC  in  each  of  these  blocks  of  instruction  (Table  E-6),  with  OSUT  soldiers 
consistently  taking  the  most  time  to  complete  a  lesson  and  exercise  and  lOBC  consistently 
requiring  the  least  time.  The  impact  of  these  times  on  the  cumulative  time  to  complete  the  code 
training  is  shown  in  Figure  12. 

Time  data  were  also  examined  for  those  soldiers  who  did  not  have  to  repeat  an  exercise. 
Although  the  times  decreased  for  those  blocks  of  instruction  when  those  requiring  remediation 
were  eliminated,  the  same  overall  effects  for  condition  and  soldier  group  remained. 
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Figure  12.  Cumulative  time  to  complete  training  on  codes,  and  times  for  each  lesson  by  soldier 
group  (includes  soldiers  who  had  to  repeat  exercises). 

Of  additional  interest  was  a  shift  in  the  relative  times  for  the  soldier  groups  within  each 
condition  on  the  rifle  and  weapons  squad  block  of  instruction.  When  all  soldiers  were  examined, 
this  interaction  approached  significance,  F(3,  159)  =  1 17.54,  p  =.0060,  Table  E-4.  As  shown  in 
Figure  13,  for  all  soldier  groups  except  lOBC,  soldiers  took  less  time  to  complete  these  blocks  of 
instruction  when  they  were  in  the  Low  Demand  condition  than  in  the  High  Demand  condition. 
For  the  lOBC  soldiers,  the  times  were  the  same.  However,  the  discrepancy  between  the  High 
and  Low  Demand  conditions  decreased  (see  Figure  13)  when  the  analysis  was  restricted  to  the 
soldiers  who  did  not  repeat  the  exercise,  since  the  times  decreased  for  the  ANCOC,  BNCOC,  and 
OSUT  groups.  In  this  case,  the  interaction  effect  did  not  approach  significance  (see  Table  E-7). 
The  “interaction  with  all  soldiers”  apparently  resulted  from  the  additional  time  required  by 
soldiers  who  had  to  repeat  exercises  in  the  High  condition,  and  the  increased  time  was  in 
proportion  to  the  number  of  soldiers  within  each  course  who  repeated  the  exercises. 

In  summary,  based  on  the  mean  scores  and  mean  times,  more  time  was  required  when 
performance  was  low.  For  example,  the  score  on  the  rifle/weapons  squads  instruction  was 
significantly  lower  for  the  High  Demand  condition  than  the  Low  Demand  condition,  and  soldiers 
in  the  High  Demand  condition  took  significantly  longer  to  complete  this  section  as  well.  In 
addition,  OSUT  soldiers  often  had  the  lowest  scores,  and  typically  required  the  most  time. 
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Figure  13.  Mean  times  on  rifle  and  weapons  squad  code  training  for  soldier  group  as  a  function 
of  High  and  Low  Demand  conditions. 


Map  Training  Results 

With  the  map  training,  the  primary  interest  was  in  determining  if  there  were  differences 
between  the  High  and  Low  Demand  conditions  on  the  exercises,  and  if  there  were  differences 
among  the  High,  Low,  and  Exploratory  conditions  on  the  final  exam.  As  with  the  codes,  course 
differences  and  interactions  with  courses  were  also  of  interest.  The  criterion  variables  were 
scores  on  the  exercises  and  the  final  exam,  and  time  to  complete  the  training.  Although  soldiers 
were  not  required  to  repeat  an  exercise  if  they  did  not  achieve  an  80%  score,  the  number  of 
soldiers  who  did  not  achieve  this  level  was  examined. 

Comparisons  were  made  between  High  and  Low  Demand  conditions  on  the  following 
exercises: 

■  Zoom,  Pan  and  Find  Me 

■  Find  X 

■  Display 

As  described  in  the  Method  section,  the  difference  between  the  High  and  Low  conditions  was 
that  soldiers  in  Low  Demand  went  through  each  exercise  immediately  after  receiving  the 
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corresponding  instruction.  Those  in  High  Demand  received  all  three  blocks  of  instruction  and 
then  had  all  the  exercises. 

Scores  on  map  exercises  and  map  final  exam.  A  two-way  analysis  of  variance  (training 
condition  by  soldier  group)  was  conducted  on  each  exercise  score  and  the  map  final  exam  score 
(see  Table  E-10).  There  were  main  effects,  but  no  significant  interactions.  There  were  two 
significant  effects  for  training  condition,  and  two  effects  for  soldier  group.  Means  for  each 
training  condition  are  in  Table  E-1 1 .  The  means  reflect  the  first  attempt  at  the  exercise  as  no 
repeats  were  allowed  because  of  time  constraints.  Final  exam  scores  include  the  Exploratory 
condition. 

Due  to  time  constraints  in  conducting  the  experiment,  not  all  soldiers  were  able  to 
complete  the  map  training.  For  Low  Demand,  three  soldiers  did  not  finish  the  training.  For  High 
Demand,  five  soldiers  did  not  finish  the  training.  The  actual  numbers  completing  each  exercise 
and  the  map  final  exam  are  shown  in  Table  E-1 1 . 

Significant  differences  occurred  for  training  condition  on  the  display  exercise  and  the 
map  final  exam,  F(l,  70)  =  4.07,;?  =  .0474  and  F{2, 142)  =  6.84,p  =  .0014,  respectively.  These 
are  shown  graphically  in  Figure  14.  On  the  display  exercise,  soldiers  in  the  Low  Demand 
condition  scored  higher  than  those  in  the  High  Demand  condition.  On  the  final  exam,  clearly, 
soldiers  in  the  Exploratory  condition  had  the  lowest  scores,  although  the  difference  between  Low 
and  High  Demand  conditions  was  also  significant.  The  fact  that  no  remedial  training  was 
provided  for  the  Low  and  High  Demand  conditions  (vice  the  symbols  training)  may  have 
affected  scores  for  soldiers  in  these  conditions  on  the  final  exam.  The  overall  downward  trend  of 
scores,  shown  in  Figures  14  and  15,  may  simply  represent  the  cumulative  effect  of  working  at  a 
computer  for  several  hours  learning  new  information. 

Means  for  the  four  soldier  groups  on  the  three  map  exercises  and  map  final  exam  are  in 
Figure  15  and  Table  E-12.  The  map  exercise  means  are  based  on  only  the  Low  and  High 
Demand  conditions,  while  the  final  exam  mean  includes  the  Exploratory  condition  as  well. 
Significant  differences  among  soldier  groups  occurred  on  the  find  X  exercise  [7^(3,  71)  =  2.%6,p 
=  .0431],  and  the  final  exam  [^(3, 142)  =  3.29,;?  =  .0226].  With  find  X,  BNCOC  did  better  than 
ANCOC  and  OSUT.  On  the  final  exam,  lOBC  did  better  than  BNCOC  and  OSUT.  Thus,  on 
each  measure,  OSUT  scored  lower  than  the  top  group. 

On  the  display  exercise,  the  difference  among  courses  approached  significance,  F(3,  70) 

=  2.52,  p  —  .0655.  There  was  considerable  variability  in  scores  in  BNCOC  and  ANCOC,  which 
could  have  contributed  to  the  lack  of  an  effect  at;?  <  .05. 


40 


100 


95 

90 


85 


o 

s 

o 

o 


80 

75 


Q> 


70 

65 


60 


55 


50 


Low  Demand 

I 

High  Demand 

Exploratory 


100 

95 

90 

85 

80 

75 

70 

65 

60 

55 

50 


Zoom/Pan  FindX  Display  Final  Exam 

Map  Exercises  and  Final  Exam 


Figure  14.  Mean  scores  for  map  exercises  and  map  final  exam  for  High  Demand,  Low  Demand, 
and  Exploratory  conditions.  (Scores  with  significant  differences  among  training  conditions  are 
highlighted.) 
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Figure  15.  Mean  scores  on  map  exercises  and  final  exam  for  each  soldier  group.  (Scores  with 
significant  soldier  group  differences  are  highlighted.) 
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Of  interest  is  the  lack  of  a  significant  interaction  between  soldier  group  and  training 
condition  on  the  final  exam.  From  high  to  low  within  each  soldier  group,  soldiers  in  the  Low 
Demand  condition  had  the  highest  scores,  those  in  the  High  Demand  condition  were  next,  and 
those  in  the  Exploratory  condition  had  the  lowest  scores.  Thus  the  training  conditions  had  the 
same  relative  effect  on  soldiers  from  all  courses  on  the  map  final  exam.  It  should  be  noted, 
however,  that  for  ANCOC^  the  relative  order  of  the  High  and  Low  Demand  conditions  was 
reversed;  but  Exploratory  scores  were  the  lowest  as  was  the  case  in  the  other  courses. 

Although  soldiers  were  not  required  to  repeat  map  training  if  they  scored  less  than  80% 
on  an  exercise,  the  number  of  soldiers  below  this  level  was  tabulated  to  see  if  there  were 
differences  between  the  High  and  Low  Demand  training  conditions.  These  results  are  in  Table 
12.  They  support  the  expectation  that  performance  would  decrease  when  individuals  were 
required  to  master  a  large  amount  of  information  within  a  block  of  instruction.  The  High 
Demand  condition  showed  the  most  impact  on  the  display  exercise.  Information  on  the  display 
function  was  the  last  to  be  presented  in  the  single  block  of  instruction  for  the  High  Demand 
condition^.  Thus  it  was  not  unexpected  that  at  this  point  in  training  soldiers  were  “overloaded” 
with  information,  which  had  a  negative  impact  upon  performance.  In  terms  of  the  soldier 
groups,  OSUT  soldiers  were  more  likely  to  score  below  80%  on  the  exercises  than  the  other 
groups.  On  average  33%  of  OSUT  soldiers  scored  below  80%  compared  to  1 1%  to  22%  for  the 
other  groups.  We  also  examined  the  percentage  of  soldiers  who  scored  below  80%  on  the  map 
final  exam.  This  percentage  was  41%  for  High  Demand,  45%  for  Low  Demand,  and  75%  for 
Exploratory. 

Table  12 

Number  of  Soldiers  who  did  not  Achieve  a  Score  of  80%  Correct  on  the  Map  Exercises 


Condition 

Exercise  (in  order  of  presentation) 

Zoom/Pan/Find  Me 

FindX 

Display 

Low  Demand 

1  of  43  (2%) 

8  of  42  (19%) 

11  of  42  (26%) 

High  Demand 

5  of  39  (13%) 

9  of  37  (24%) 

16  of  36  (44%) 

Note.  Soldiers  in  High  Demand  took  these  three  exercises  after  receiving  all  map  instruction. 
Soldiers  in  the  Low  Demand  took  the  exercises  immediately  after  the  corresponding  lesson. 


Times  on  map  exercises  and  map  final  exam.  Time  to  complete  the  lessons,  exercises, 
and  final  exam  was  also  examined.  A  two-way  ANOVA  (map  training  condition  by  soldier 
group)  was  conducted  on  each  block  of  instruction,  the  map  final  exam,  map  training,  and  total 
time  in  the  map  phase  (Table  E-13).  There  were  main  effects  but  no  interactions.  Illustrations  of 
the  time  results  are  in  Figure  16.  Means  are  in  Tables  E-14  and  E-15. 

Several  significant  differences  occurred  between  the  training  conditions  on  map  times. 
First,  the  soldiers  in  the  Low  Demand  condition  were  significantly  faster  than  those  in  High 
Demand  in  the  zoom/pan  block  of  instruction/exercises,  F(1 ,  74)  =  5.34,  p  =  .0235,  and  in  the 
time  to  complete  the  range/azimuth  instruction,  F(l,  75)  =  29.87,/?  =  .0000. 


^  This  change  in  order  was  not  significant,  but  is  pointed  out  for  descriptive  purposes. 

^  In  reality,  the  last  information  presented  was  on  the  range-azimuth  function,  but  there  was  no  exercise  to  assess 
proficiency  on  this  task. 
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There  were  also  differences  when  the  time  devoted  to  all  training  (lessons  and  exercises 
and  exploring  the  map)  was  examined.  Here,  High  and  Low  Demand  condition  times  were  four 
times  slower  than  Exploratory  times,  F(2, 143)  =  622.65,  p  =  .0000.  On  average,  soldiers  in  the 
Exploratory  condition  spent  12.5  min  working  with  (learning  about)  the  map  fvuictions  on  their 
ovra.  The  fastest  time  was  2  min;  the  longest  was  35  min  (Table  E-14).  This  compares  to 
averages  of  54  min  for  the  soldiers  in  the  Low  and  High  Demand  conditions  to  complete  all  the 
lessons  and  exercises. 

Although  the  soldiers  in  the  Exploratory  condition  completed  their  instruction  very 
quickly,  they  took  more  time  to  complete  the  map  final  exam  than  the  other  two  conditions  (22 
min  vs.  15  min,  see  Table  E-14).  Exploratory  times  on  the  final  exam  were  significantly  slower 
than  both  the  Low  and  High  Demand  times,  F{2, 138)  =  46.43,/?  =  .0000.  As  illustrated  in 
Figure  16,  this  increased  amount  of  time  was  the  same  for  all  soldier  groups.  It  appeared  that 
soldiers  in  the  Exploratory  condition  tried  to  make  up  for  what  they  did  not  learn  when  taking  the 
exam  itself. 

Of  additional  interest  is  that  data  on  the  final  exam  indicate  the  Exploratory  group  spent 
more  time  reading  and  studying  the  questions  than  the  other  two  groups.  The  map  exam  had  22 
items  each  with  a  time  limit  of  one  minute.  The  maximum  times  in  the  High  and  Low  Demand 
conditions  were  20  minutes,  meaning  that  all  soldiers  within  these  conditions  completed  all  items 
plus  all  instructions  within  20  minutes.  However,  the  maximum  time  for  the  Exploratory 
condition  was  42  minutes,  with  59%  of  the  soldiers  (47  of  79)  taking  more  than  20  minutes.  The 
only  opportunity  for  additional  time  during  the  exam  was  in  examining  the  questions  prior  to 
officially  answering  a  question.  It  is  not  known  what  soldiers  were  doing  while  examining  the 
questions  —  were  they  trying  to  interpret  or  imderstand  the  question  or  to  determine  their 
strategy  in  how  to  answer  it?  Regardless,  this  increased  time  reflects  a  lower  level  of  proficiency 
for  the  Exploratory  condition. 

There  were  also  significant  differences  in  time  among  the  soldier  groups.  They  differed 
on  time  to  complete  three  different  blocks  of  map  training:  zoom/pan,  F(3,  74)  =  5.72,  p  = 

.0014;  find  X,F(3,  71)  =  4.50,/?  =  .0059;  and  display ,  F(3,70)  =  9.28,/?  =  .0000.  On  all  three 
blocks  of  instruction  (zoom/pan,  find  X,  display),  lOBC  soldiers  were  consistently  faster  and 
OSUT  consistently  slower.  In  addition,  ANCOC  was  significantly  faster  than  OSUT  on  find  X 
and  display.  lOBC  was  significantly  faster  than  BNCOC  and  ANCOC  on  zoom/Pan,  and  faster 
than  BNCOC  on  display  (Tables  E- 13  and  E-15).  On  the  time  to  complete  training  (lessons  and 
exercises  for  the  High  and  Low  Demand  groups  and  the  time  working  with  the  map  for 
Exploratory),  lOBC  was  significantly  faster  than  BNCOC  and  OSUT,  F(3, 143)  =  5.55,  p 
=.0000.  The  same  pattern  occurred  on  the  total  time  spent  on  the  map  phase  of  training.  lOBC 
was  significantly  faster  than  BNCOC  and  OSUT,  F(3,  138)  =  7.66,  p  =.0000. 

In  summary  and  as  illustrated  in  Figure  16,  the  major  impact  on  time  was  the  training 
conditions,  although  soldier  groups  differed  significantly  on  the  map  times  as  well.  Soldiers  in 
the  Exploratory  group  did  not  explore  very  long,  but  spent  more  time  on  the  exam  than  soldiers 
in  the  other  two  training  conditions. 
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Map  Cumulative  Times  in  Minutes 
Training  and  Final  Exam  Timas  for  All  Map  Conditions 


75 


The  map  training  times  show  that 
Exploratory  soldiers  spent  12  minutes 
working  with  the  map  during  training, 
significantly  less  time  than  the  54 
minutes  for  those  in  High  and  Low 
Demand.  However,  those  in  High  and 
Low  conditions  spent  significantly  less 
time  in  the  map  final  (15  vs.  22 
minutes). 


Over  both  Low  and  High  Demand 
conditions,  lOBC  soldiers  finished 
training  significantly  faster  than 
OSUT  soldiers. 


Distribution  of  total  map  times  by  training 
condition  shows  where  the  middle  50%  of 
the  times  were  concentrated  for  each 
condition,  as  well  as  the  extreme  times.  The 
minimal  overlap  between  Exploratory  times 
and  High  and  Low  Demand  times  is  clear. 


lOBC 

ANCOC 

BNCOC 

OSUT 


Zoom/Pan  Find  X  Display  Range/ 

Azimuth 


0  10  20  30  40  50  60 

Map  Cumulative  Training  Time  in  Minutes  for  Low  and  High  Conditions 


Times  on  the  map  final  exam  show 
that  Exploratory  soldiers  in  each 
soldier  group  took  about  5  minutes 
more  on  the  exam  than  soldiers  in 
High  and  Low  Demand  conditions. 


Figure  16.  Illustrations  of  time  spent  on  the 
map  exercises  and  map  exam. 
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Correlations  among  Measures 

Scores  and  times.  The  relationship  among  the  exercise  and  final  exam  scores  and  times 
to  complete  the  exercises  and  training  was  a  primary  question  for  both  the  code  and  map 
training.  The  correlation  between  the  two  final  exams,  for  all  participants,  was  .23  {p  <  .001), 
significant,  but  not  substantial. 

In  summary,  for  both  the  code  and  map  phases  of  training,  the  typical  relationships 
among  scores  and  times  were  as  follows; 

•  Exercise  and  final  exam  scores  correlated 

•  Times  correlated  with  each  other 

•  Scores  correlated  negatively  with  times  (higher  scores  were  associated  with  faster 
times). 

Specific  results  for  the  code  and  map  phases  of  training  are  discussed  in  turn. 

Code  correlations.  Product  moment  correlation  coefficients  among  all  the  code  scores 
(exercises  and  final  exam)  were  calculated,  a  total  of  15.  Nine  were  significant  (Table  E-16). 
The  highest  correlations  were  between  rifle/weapons  squads  and  leader/unit,  and  between 
leader/unit  and  the  exam,  each  r  =  .43).  Other  significant  correlations  ranged  from  .15  to  .37. 
One  exercise  score,  the  initial  symbol  exercise  (weapon  and  unit  symbols),  did  not  correlate  with 
any  other  exercises  nor  with  the  final  exam.  This  could  have  been  because  soldiers  were  given 
inamediate  feedback  on  their  answers  and  had  an  opportunity  to  learn  the  symbols  as  a  function 
of  the  feedback  during  this  exercise.  Each  symbol  was  presented  three  times,  so  corrective 
feedback  could  have  improved  soldier  performance  during  the  exercise  itself. 

Correlations  among  the  times  to  complete  the  code  blocks  of  instruction  (lessons  and 
exercises)  and  the  code  final  exam  were  also  computed,  a  total  of  15.  Each  correlation  was 
significant  (Table  E-17).  In  addition,  the  times  for  each  of  these  blocks  of  code  training  were 
correlated  to  the  total  code  time.  These  correlations  were  also  significant.  Of  the  21 
intercorrelations  that  were  computed,  12  were  at  least  .50,  with  the  highest  being  .86 
(rifle/weapons  squad  time  with  final  exam  time).  In  general,  the  magnitude  of  the  correlations 
among  the  code  time  measures  was  stronger  than  among  the  code  scores. 

For  code  training,  separate  lesson  and  exercise  times  were  not  available.  Only  the  total 
time  to  complete  each  lesson  and  exercise  combination  was  recorded  in  the  database.  For  each 
block  of  instruction,  the  correlation  between  score  and  time  was  both  negative  and  significant, 
meaning  higher  scores  were  associated  with  faster  times  (see  correlations  on  the  diagonal  in 
Table  E-18).  These  correlations  were  considered  moderate,  with  four  of  the  five  correlations 
ranging  from  -.32  to  -.46.  The  correlation  between  the  code  final  exam  score  and  code  final 
exam  time  was  -.15,  also  significant  but  small. 

Map  correlations.  For  the  map  scores  and  times,  the  Low  and  High  Demand  conditions 
were  examined  separately  from  the  Exploratory  condition,  as  the  Exploratory  condition  did  not 
have  lessons  or  exercises.  In  contrast  with  the  code  training,  separate  times  for  the  map  lessons 
and  exercises  were  available. 
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As  with  codes,  the  map  exercise  scores  and  final  exam  score  correlated  significantly  with 
each  other.  The  six  correlations  ranged  from  .31  to  .61  (see  Table  E-19).  Correlations  among 
the  map  times  (Table  E-20),  computed  separately  for  lessons  and  exercises,  were  all  significant. 
But  they  were  typically  higher  than  the  map  score  intercorrelations  ( 1 2  of  the  1 6  correlations 
were  higher  than  .50). 

Of  considerable  interest  was  that  the  times  on  the  map  lessons  did  not  correlate  with 
scores  (Table  E-21).  On  the  other  hand,  time  spent  on  the  map  exercises  correlated  significantly, 
but  negatively  with  map  exercise  scores.  The  correlations  for  Zoom/Pan,  Find  X,  and  Display 
ranged  from  -.35  to  -.43.  In  summary,  high  map  exercise  scores  were  indicative  of  greater  speed 
on  the  exercise  itself,  but  did  not  relate  to  time  spent  acquiring  knowledge  of  the  map  functions. 

For  the  Exploratory  map  condition,  there  was  a  significant  but  small  relationship  (r  =  .26) 
between  the  final  exam  score  and  the  time  spent  exploring  (Table  E-22).  There  was  no 
relationship  between  the  map  final  score  and  the  time  to  take  the  final  exam. 

Scores  and  background  variables.  Of  particular  interest  was  whether  any  of  the 
measures  of  military  knowledge,  computer  experience,  and  tendency  to  work  independently 
correlated  with  the  two  final  exam  scores.  As  cited  previously,  the  1 2  background  variables 
were  grouped  into  the  three  categories  listed  below. 

♦Computer  background  from  survey 
-Computer  ownership 

-Frequency  with  which  soldiers  used  key  computer  features 
-Self-rating  of  expertise 
-Icon  score 

♦Military  knowledge 

-Score  on  weapon  symbols 
-Score  on  unit  symbols 
-Score  on  map  reading 
-Prior  use  of  battle  roster  system 
-Training  on  the  battle  roster  system 

♦Tendency  to  solve  problems  on  own 

-Score  on  tendency  to  work  independently  scale 

-Self-rating  on  tendency  to  develop  computer  shortcuts 

-Self-rating  on  tendency  to  have  a  “knack”  in  working  with  computers 

A  principal  component  analysis  of  these  measures  was  conducted  prior  to  regression 
analyses  on  the  exam  scores.  The  purpose  of  this  analysis  was  to  determine  if  any  of  the 
background  variables  could  be  combined  for  the  regression  analysis.  Factor  scores  were  not 
used  in  the  regression  analyses,  because  it  was  of  more  interest  to  examine  the  contribution  of 
the  “original  scores”  rather  than  a  weighting  of  scores  within  a  test  battery  that  could  change 
with  time.  The  principal  components  analysis  showed  four  components,  accounting  for  67%  of 
the  variance  (see  Table  13),  and  partially  supported  the  original  grouping  of  measures  cited 
above.  As  described  below,  it  was  possible  to  combine  some  variables  (add  scores),  reducing  the 
predictors  in  the  regression  analysis  from  12  to  9. 
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Table  13 

Factor  Analysis  (principal  components)  Results  on  Background  Variables 


1 

1 

Component 

1 

“Computer” 

2 

“BR” 

3 

“Military 

Knowledge” 

4 

“Work 

Independently' 

Eigenvalue 

3.78 

1.66 

1.54 

1.01 

%  Variance 

31.52 

13.83 

12.84 

8.34 

Factor  Loadings 

Own  Computer 

.67 

.24 

.12 

.34 

Frequency  of  Use 

.77 

-.04 

-.12 

.35 

Self-rate  Skill 

.77 

-.14 

.06 

.10 

Icon  Score 

.73 

-.16 

.01 

-.01 

Knack  for  Computers 

.80 

-.14 

-.26 

-.05 

Use  Shortcuts  w/ 

.79 

-.10 

-.31 

-.03 

Computers 

.71 

-.22 

Weapon  Symbols  Score 

.33 

-.07 

Unit  Symbols  Score 

.45 

.28 

.53 

-.35 

Map  Reading  Score 

.03 

.48 

.47 

-.02 

BR  Use 

.09 

.79 

-.27 

.07 

BR  Training 

.14 

.77 

-.26 

-.09 

Work  Independently 

-.16 

.02 

.44 

.75 

•  't  A 

Note.  Factor  loadings  that  were  at  least  .70  are  bolded  and  were  used  to  characterize  each  factor. 


The  first  and  main  component  clearly  reflected  computer  use  and  perceived  skill,  from 
the  computer  survey  as  well  as  the  knack  and  short  cut  items,  which  dealt  specifically  with  a 
computer.  It  was  not  possible  to  combine  the  computer  survey  indices,  due  to  the  different  scales 
and  measurement  procedures  for  each.  However,  the  1 0-point  rating  scales  for  knack  and 
shortcut  were  summed  (bivariate  correlation  between  these  two  ratings  was  .76,  see  Table  E-23). 
Thus  it  appeared  that  the  knack  and  shortcut  variables  reflected  computer  expertise  and 
backgroimd  more  than  they  reflected  a  general  tendency  to  work  things  out  independently. 

The  second  factor  reflected  experience  with  the  BR  system  and  these  two  items  were 
combined  (summed,  bivariate  correlation  was  .49).  The  symbols  and  map  reading  scores  tended 
to  load  on  several  factors.  However,  it  was  decided  to  combine  the  weapon  and  unit  symbol 
scores,  yielding  one  score  for  knowledge  of  military  symbols  (bivariate  correlation  was  .42). 

The  work  independently  measure  appeared  to  be  relatively  independent  of  the  other  measures, 
loading  most  heavily  on  the  fourth  component.  In  sum,  for  the  regression  analysis,  the  original 
12  background  variables  were  reduced  to  9  variables  listed  below. 

♦Computer  ovraership 

♦Frequency  with  which  soldiers  used  key  computer  features 

♦Self-rating  of  computer  expertise 


♦Computer  icon  score 

♦Short/Knack  (Self-rating  on  tendency  to  develop  computer  shortcuts  plus 

Self-rating  on  tendency  to  have  a  “knack”  in  working  with  computers) 

♦BR  experience  (Prior  use  of  BR  system  plus  Training  on  the  BR  system) 

♦Symbols  (Score  on  weapon  symbols  plus  Score  on  unit  symbols) 

♦Map  reading  score 

♦Tendency  to  work  independently  scale 

The  relationships  between  the  nine  background  variables  and  the  two  final  exam  scores 
were  examined  separately  for  each  of  the  training  conditions  for  the  codes  and  the  map  functions. 
Initially,  all  background  variables  were  entered  into  the  multiple  regression  to  establish  the 
maximum  value  of  the  multiple  R.  Next  a  forward  stepwise  regression  was  conducted,  to 
determine  which  variables  contributed  the  most  to  the  prediction.  The  regression  results  are 
documented  in  Table  E-25.  The  multiple  Rs  for  the  stepwise  solutions  ranged  from  .33  to  .47, 
with  the  highest  R  for  the  Exploratory  condition  on  the  map  final  exam  (see  Table  14).  Results 
showed  that  only  measures  from  the  computer  and  the  military  knowledge  dimensions 
contributed  to  the  prediction  of  the  final  exam  scores.  The  BR  and  work  independently  measures 
did  not  predict. 

The  Exploratory  map  condition  where  soldiers  learned  on  their  own  was  of  particular 
interest.  Could  we  predict  what  type  of  soldier(s)  was  (were)  best  suited  for  an  Exploratory 
training  condition?  The  computer  icon  score  was  the  strongest  correlate  for  the  Exploratory 
condition  (r  =  .42),  followed  by  military  knowledge  of  symbols,  although  this  score  did  not 
increase  the  R  substantially  (see  Table  14).  Perhaps,  experience  with  computers  made  it  easier 
for  soldiers  to  know  what  and  how  to  examine  the  interface  on  their  own.  The  tendency  to  work 
independently  scale  did  not  correlate  with  the  map  score  for  the  Exploratory  group,  even  though 
it  was  developed  for  that  purpose. 

Table  14 

Stepwise  Multiple  Regression  Results 


Criterion  Measures  and  Training  Conditions 

Code  Final  Exam 

Map  Final  Exam 

Low 

(«  =  80) 

High 

{n  =  83) 

Low 

(n  =  40) 

High 

(«  =  34) 

Exploratory 

{n  =  80) 

Predictor  R 

Predictor  R 

Predictor  R 

Symbols  .26 
Map  Read  .33 

Icon  Score  29 
Map  Read  .41 

Short/Knack  .38 

Map  Read  .36 
Self-rate  .46 

Icon  Score  .41 
Symbols  .47 

R 

.33 

.41 

.38 

.46 

A1 

Note.  Probability  of  F  to  enter  was  set  at .  1 0.  Probability  of  F  to  remove  was  set  at .  1 5. 
Variables  are  listed  in  order  of  their  entry.  Resulting  R  value  is  also  shown  at  each  step. 


In  general,  the  stepwise  Rs  were  not  high  (.33  to  .47),  nor  were  the  full  equation  Rs  (.42 
to  .52).  For  each  exam,  the  R  for  Low  Demand  was  lower  than  that  for  High  Demand.  With  the 
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exception  of  the  code  exam  score  for  Low  Demand,  some  measure  of  computer  expertise  or 
background  was  in  the  stepwise  regression  equations.  With  the  exception  of  the  map  exam  for 
Low  Demand,  the  score  on  symbols  or  map  reading  (military  knowledge)  was  in  the  stepwise 
regression.  Battle  roster  experience  never  correlated  highly,  perhaps  because  so  few  soldiers  had 
experience  with  the  BR  system. 


Discussion  and  Conclusions 

Despite  substantial  differences  in  military  knowledge  and  experience  among  the  soldier 
groups  as  well  as  the  minimal  knowledge  in  some  areas  by  all  groups,  neither  these  differences 
nor  the  limited  knowledge  appeared  to  greatly  hinder  performance.  For  instance,  soldiers 
learned  the  weapon  and  unit  symbols  even  though  many  did  not  attempt  to  answer  these 
questions  on  the  initial  military  knowledge  test  as  they  had  never  been  exposed  to  this  material  in 
their  military  training. 

For  both  the  code  and  map  training,  the  blocks  of  instruction  that  had  the  greatest 
differences  in  working  memory  demand  were  the  ones  that  resulted  in  differences  in 
performance.  For  the  coding  training,  this  was  the  rifle  and  weapons  squads  block(s)  Of 
instruction.  In  the  map  training,  this  difference  was  ultimately  reflected  in  performance  on  the 
Display  exercises.  In  each  instance,  those  soldiers  in  the  Low  Demand  condition  performed 
better. 


Another  factor  that  indicates  the  Low  Demand  condition  was  more  effective  than  High 
Demand  was  the  variability  in  scores.  We  checked  for  conditions  where  the  variability  of  scores 
for  a  given  group  was  at  least  1.5  times  greater  than  another.  For  both  the  company-squad  BR 
and  rifle-weapons  squad  code  exercises,  the  scores  for  the  High  Demand  condition  were  at  least 
2.5  times  more  variable  than  those  for  the  Low  Demand  condition.  Thus  not  only  did  the  Low 
working  demand  condition  result  in  a  higher  level  of  performance  than  the  High  condition,  it  also 
resulted  in  more  consistent  scores. 

Those  individuals  using  the  unguided  exploratory  means  of  learning  the  map  performed 
more  poorly  than  those  with  more  formal  instruction.  However,  the  time  devoted  to  exploratory 
learning  was  70%  less  than  that  in  the  Low  and  High  Demand  conditions.  Exploratory  times 
were  at  least  twice  as  variable  as  those  in  the  Low  and  High  Demand  conditions,  and  the  time 
required  to  complete  the  map  final  exam  was  longer.  Both  findings  were  additional  indications 
of  lower  proficiency.  In  summary,  exploratory  learning,  with  minimal  guidance,  was  not  the 
most  effective  mode  of  digital  map  training  for  the  diversity  of  soldiers  in  the  research  -  the 
soldiers  to  receive  the  Land  Warrior  system.  Although  the  tendency  to  work  independently  scale 
did  seem  to  identify  soldiers  who  tended  to  work  without  guidance,  it  did  not  necessarily  identify 
those  who  were  good  at  employing  the  appropriate,  information-seeking  strategies  on  their  own. 

Differences  in  scores  and  times  for  the  four  soldier  groups  appeared  on  many  measures. 
Typically,  lOBC  soldiers  scored  the  highest,  then  ANCOC,  then  BNCOC,  and  lastly  OSUT. 

That  OSUT  soldiers  scored  lowest  was  not  unexpected,  given  their  limited  military  experience 
and  knowledge.  Times  also  varied  with  soldier  group,  although  the  pattern  was  not  as  consistent 
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as  was  the  case  for  the  scores.  But  in  general,  lOBC  soldiers  finished  quickest,  and  OSUT 
soldiers  took  the  most  time.  As  indicated  by  the  time  data  and  observations  made  during  the 
experiment,  the  CBT  format  adapted  well  to  the  differences  in  learning  rate  exhibited  by  the 
soldiers.  The  experiment  simply  reinforces  findings  from  previous  CBT  research  about  the 
efficiency  of  this  training  approach,  as  it  avoids  the  lock-step  process  inherent  in  teacher-led 
classroom  instruction.  This  increased  efficiency  can  be  viewed  as  particularly  valuable  when  the 
target  user  population  is  quite  diverse,  as  it  is  with  many  of  the  Army’s  digital  systems. 

Of  interest  was  the  lack  of  statistical  interactions  between  the  experimental  conditions 
and  the  soldier  groups.  This  was  not  expected.  Prior  research  (Como  &  Snow,  1986)  indicates 
that  high  ability  and/or  or  highly  motivated  individuals  perform  well  in  discovery  learning 
contexts  or  in  situations  where  they  have  some  degree  of  control,  whereas  less  able  individuals 
require  more  teacher  direction  and  instmctional  support.  Although  we  had  no  measure  of  soldier 
ability  per  se,  the  soldier  groups  did  differ  substantially  in  military  experience  and  knowledge. 
Consequently,  it  seemed  reasonable  for  the  High  Demand  conditions  to  have  a  greater  impact  on 
OSUT  soldiers  than  the  other  groups,  given  the  limited  military  background  of  trainees.  Instead, 
when  the  High  Demand  condition  impacted  scores,  it  had  a  similar  detrimental  effect  on  each 
group.  Also  it  seemed  plausible  that  the  lOBC  soldiers  might  do  as  well  under  an  Exploratory 
condition  as  the  other  conditions,  given  their  university  education.  Although  they  were  the 
highest  performing  soldier  group  imder  this  condition,  their  performance  dropped  relative  to  their 
performance  in  the  Low  and  High  Demand  conditions.  A  similar  pattern  of  results  occurred  for 
the  other  groups. 

A  very  practical  question  is  whether  any  of  the  training  conditions  resulted  in  satisfactory 
levels  of  performance.  The  criterion  was  80%  correct.  For  codes,  soldiers  in  both  conditions 
averaged  over  80%  correct  on  the  code  final  exam.  The  averages  for  each  course  were  greater 
than  80%  as  well.  A  factor  that  could  have  contributed  to  soldiers  meeting  the  code  criterion  on 
the  final  exam  was  that  they  had  to  repeat  a  block  of  instruction  if  they  did  not  achieve  80%  on 
the  exercises  the  first  time. 

For  the  map  exam,  the  final  level  of  performance  was  not  as  high,  falling  below  80%  for 
each  training  condition  and  for  each  group  of  soldiers.  There  was  no  requirement  for  soldiers  in 
the  Low  and  High  Demand  conditions  to  repeat  map  exercises  when  they  failed  to  reach  the  80% 
criterion.  If  this  had  been  required,  it  is  likely  that  the  final  scores  in  these  two  conditions  would 
have  been  higher,  resulting  in  an  even  greater  discrepancy  with  the  Exploratory  condition. 

A  second  practical  question  that  remains  imanswered  in  the  present  research  is  whether 
the  CBT  lessons  and  exercises  had  a  lasting  effect.  This  issue  could  not  be  addressed  in  the 
context  of  the  current  effort. 

Given  the  relatively  short  time  soldiers  spent  “exploring”  the  map  interface  and  the  small 
but  significant  relationship  between  exploratory  time  and  the  final  exam  score,  exploratory 
modes  of  training  bear  filler  investigation.  Exploratory  learning  should  not  be  automatically 
rejected  as  a  training  approach  on  the  basis  of  this  research.  However,  the  results  do  support 
prior  research  on  the  limitations  of  “pure”  or  “unguided”  exploratory  modes  of  instruction. 
Combining  some  elements  of  formal  instruction  with  an  exploratory  mode  might  prove  very 
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effective  in  acquiring  the  interactive  skills  and  insights  required  to  work  with  digital  interfaces 
typical  of  Army  systems.  Some  form  of  guided  exploratory  learning  could  also  be  a  good  way  of 
adapting  to  individual  differences  in  groups  of  soldiers  with  diverse  military  backgrounds. 

Not  unexpectedly,  you  can  include  too  much  information  in  a  block  of  instruction.  The 
concept  of  the  number  seven  can  be  used  as  a  rough  guide  to  determine  when  too  much  new 
information  may  be  presented.  And  it  should  not  be  viewed  as  a  formula  for  instructional  design. 
As  pointed  out  by  Ausubel  (1968),  blocks  of  instruction  should  not  be  made  small  just  to  reduce 
the  error  rate.  They  should  not  fragment  content,  ignore  the  need  to  consolidate  and  integrate 
information,  or  fail  to  provide  individuals  with  the  logic  of  the  body  of  content  that  is  being 
taught.  In  addition,  chunks  of  meaningful  material  can  easily  vary  with  the  background  of  the 
learner.  Thus  large  bodies  of  material  may  be  easily  comprehended  by  some  individuals  when 
familiar  information  is  included,  but  be  overwhelming  for  others. 

The  experiments  showed  how  the  computer-based  training  could  be  designed  to 
incorporate  military  tactical  system  software  as  background  instruction  and  demonstration 
screens,  and  as  interactive  screens  for  performance  exercises.  High-fidelity  training  is  a  positive 
by-product  of  this  technique. 

Although  the  multi-media  instruction  was  effective,  the  medium  per  se  is  not  an 
automatic  panacea  for  training  the  digital  skills  required  of  soldiers  and  leaders  who  must  use  the 
tactical  software  embedded  in  new  equipment.  Measurement  techniques  and  procedures  are 
needed  that  accoimt  for  the  multiple,  yet  valid  approaches  a  soldier  can  use  to  accomplish  a  task. 
In  addition,  challenges  lie  in  developing  problem-solving  scenarios  that  require  soldiers  to  go 
beyond  demonstrating  technical  skill  proficiency  to  demonstrating  effective  employment  of 
digital  skills  in  combat-like  settings. 
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Appendix  A 


Symbols  and  Battle  Roster  System 


individuals 


AC1110 


I AC1112 


4^ 


AC1113  I 


I  AC1241 


I  AC1245  I 


Weapon  Symbols 

Rifle  Squad  Automatic  M203  Grenade  M60  or  M240  Antitank  Weapon 

Weapon  (SAW)  Launcher  Machine  Gun  Javelin 

SAW  symbol  generated  for  purposes  of  the  experiment;  no  official  Army  symbol  existed. 


Battle  Roster:  1st  two  characters  indicate  Battalion,  1st  number  is  code  for  company,  2nd 
number  is  code  for  platoon,  3rd  number  is  code  for  squad,  4th  number  is  code  for  individual. 
Leaders  coded  as  0. 


Units 


OOP 

n 

X 

AC333 

AC12 

AC1 

Unit  Symbols 

Squad  Platoon  Company 

Battle  Roster.  Characters  and  numbers  as  cited  for  individuals.  Truncated  at  the 
appropriate  level.  This  procedure  was  developed  for  purposes  of  the  experiment;  not 
an  official  Army  procedure. 
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Appendix  B 

Pre-experiment  Instruments 


B-l 


Computer  Training  Research 
(U.S.  Army  Research  Institute,  Ft.  Benning,  GA) 

Log-In  Code: 


Name  (optional) _ 

Please  Do  Not  Begin  Until  Instructed 


PRIVACY  ACT  STATEMENT 

Public  Law  93-573,  called  the  Privacy  Act  of  1974,  requires  that  you  be 
informed  of  the  purpose  and  uses  to  be  made  of  the  information  collected. 

The  Department  of  the  Army  may  collect  the  information  requested  in  this 
questionnaire  under  the  authority  of  10  USC  2358.  Providing  Information  in 
this  questionnaire  is  voluntary.  Failure  to  respond  to  any  particular  questions 
will  not  result  in  penalty. 


This  packet  contains  the  paper  and  pencil  portion  of  the  Computer  Training 
Study.  The  package  is  divided  into  three  sections: 

Section  A:  Survey  on  computer  experience 

Section  B;  Survey  on  problem  solving  behavior 

Section  C;  Quiz  on  map  reading,  symbols,  and  battle  roster  system 


•  We  will  provide  instructions  for  the  computer  training  portion  of  the  study 
once  everyone  has  completed  Sections  A  through  C.  Please  do  not 
attempt  to  log  onto  the  computer  until  instructed  to  do  so. 
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Section  A 

COMPUTER  SURVEY 


Age; _  Rank: _ 

Years  and  Months  in  Army:  _ ^years  _ months 

Are  you  returning  to  the  same  position  you  held  when  you  left  your  unit?  Yes _  No 

If  Yes,  what  is  that  position?  _ 

If  No,  what  position  are  you  going  to?  _ _ 


1.  When  did  you  use  computers  in  your  education?  (Circle  all  that  apply) 

Grade  School  Jr  High  High  School  Technical  School  College  Did  Not  Use 

2.  Where  do  you  currently  use  a  computer?  (Circle  all  that  apply) 

Home/barracks/BOQ  UnitAYork  Site  Library/Learning  Ctr/Training  Facility  Do  Not  Use 


3.  For  each  of  the  following  questions,  circle  the  response  that  best  describes  you. 


a.  Do  you  own  a  personal  computer?  Yes  No 


b.  How  often  do  you:  (circle  how  frequently  you  use  each) 
•Use  a  mouse? 

•Play  computer  games? 

•Use  icon-based  programs/software? 

•Use  programs/software  with  pull-down  menus? 

•Use  graphics/drawing  features  in  software  packages? 
•Use  E-mail  (at  home  or  at  work)? 

•Use  the  Internet? 


Daily,  Weekly, 
Daily,  Weekly, 
Daily,  Weekly, 
Daily,  Weekly, 
Daily,  Weekly, 
Daily,  Weekly, 
Daily,  Weekly, 


Monthly,  Less 
Monthly,  Less 
Monthly,  Less 
Monthly,  Less 
Monthly,  Less 
Monthly,  Less 
Monthly,  Less 


Often,  Never 
Often,  Never 
Often,  Never 
Often,  Never 
Often,  Never 
Often,  Never 
Often,  Never 


4.  Which  of  the  following  best  describes  your  typing  ability?  (check  one) 

_ Hunt  and  peck  slowly 

_ Hunt  and  peck  quickly 

_ Type  slowly  while  not  looking  at  the  keyboard 

_ Type  quickly  while  not  looking  at  the  keyboard 

5.  Which  of  the  following  best  describes  your  expertise  with  computers?  (check  V  one) 

_ Novice 

_ Good  with  one  type  of  software  package  (such  as  word  processing  or  work  calendars  or  slides) 

_ Good  with  several  software  packages 

_ Can  program  in  one  language  and  use  several  software  packages 

_ Can  program  in  several  languages  and  use  several  software  packages 

_ Expert  -  Bill  Gates  would  hire  me 


If  you  are  good  with  one  or  more  software  packages,  please  list  them. 


If  you  can  program  in  one  or  more  languages,  please  name  these  languages. 
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6.  What  is  the  function  of  the  following 


Section  B 

Survey  on  Problem  Solving  Behavior 

Below  are  15  activities  that  you  may  engage  in  from  time  to  time.  Next  to  each  item,  please  circle  the  ONE 
response  that  BEST  DESCRIBES  how  you  typically  approach  each  activity.  That  is,  do  you  USUALLY: 

1.  Complete  the  activity  (or  give  up)  without  referring  to  instructions,  a  map,  or  asking  for  directions; 

2.  Proceed  at  first  without  reading  instructions,  a  map,  or  asking  for  directions,  but  wiil  refer  to  these  if 
you  run  into  a  problem; 

3.  Read  the  instructions,  a  map,  or  ask  for  help  before  beginning; 

4.  Find  someone  else  to  perform  the  task  for  you  (a  friend,  spouse,  etc.). 


Set  up  a  new  computer 

Change  a  tire  on  a  new  vehicle 

Change  settings  on  a  new  digital  watch 

Drive  an  unfamiliar  route  to  a  new  destination  more 
than  100  miles  from  home _ 

Cook  a  frozen  dinner 

Fill  out  tax  forms 

Program  a  new  VCR 

Find  your  seat  at  a  ballgame  or  concert _ 

Play  a  new  video  or  computer  game 
Take  a  new,  over-the-counter  medication 
Put  together  a  new  bicycle 


Complete  Only  refer  to  Read  Find 

task  or  give  instructions,  instructions,  someone  else 
up  without  map,  etc.  if  map,  etc.  to  do  it 


Not 

applicable 


instructions, 
map,  etc. 


there  are 
problems 


before 

beginning 


Put  up  a  new  tent 

1 

2 

3 

4 

N/A 

Respond  to  a  survey 

1 

2 

3 

4 

N/A 

Put  together  a  piece  of  furniture  that  requires  assembly 

1 

2 

3 

4 

N/A 

Does  not 
describe 
me  at  all 


I  seem  to  have  a  “knack”  or  “feel”  for  finding  my  way 
around  a  computer  program 

Even  if  I  already  know  one  way  to  perform  a  task  on  the 
computer,  I  usually  figure  out  a  shortcut  that  will  allow  me 
to  do  the  same  task  with  fewer  steps 


Somewhat 

Descriptive 


Describes 

me 

completely 
9  10 
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Section  C 

Map  Skills,  Operational  Symbols  and  Battle  Rosters 

Directions:  Given  the  1/50,000  scale  map  In  front  of  you  and  using  a  protractor,  answer  the 
following  questions  as  accurately  as  possible. 

1 .  What  is  the  8-digit  grid  coordinate  for  the  SP?  _ 

2.  What  is  the  8-digit  grid  coordinate  for  the  OBJ? _ 

3.  What  terrain  feature  is  the  OBJ  on?  _ 

4.  If  you  traveled  on  a  grid  azimuth  of  80  degrees  from  the  SP,  what  is  the  name  of  the  first  hard 

surfaced  road  you  encounter?  _ 

5.  What  is  the  straight-line  distance  in  meters  from  the  SP  to  the  OBJ?  _ 

6.  What  is  the  grid  azimuth  in  degrees  from  the  SP  to  the  OBJ?  _ 

The  remaining  questions  do  not  require  a  map: 

8.  Draw  in  the  elements  that  will  complete  each  unit  symbol: 


a)  squad  b)  platoon  c)  company 

9.  In  the  spaces  provided,  name  the  following  weapon  systems: 


10.  Have  you  ever  received  training  on  the  Army’s  battle  roster  system?  Yes _ No_ 

1 1 .  Have  you  used  the  battle  roster  system  in  a  duty  position  within  a  unit?  Yes _ No. 
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Practice  Exercises  for  Land  Warrior  Map  Exploration 


The  following  practice  exercises  will  help  you  familiarize  yourself  with  the  Land  Warrior  map  functions. 
This  is  NOT  a  test,  and  you  will  not  be  scored  on  your  responses. 

In  addition  to  completing  the  tasks  listed  below,  continue  practicing  all  available  functions.  Do  not 
proceed  to  the  Final  Exam  until  you  feel  you  are  ready  to  perform  a  series  of  complex,  timed  exercises. 

Practice  Exercises: 

1 .  Pan  around  the  map  display  until  you  locate  Gow  Hill. 

2.  Zoom  out  until  BM  (Benchmark)  147  is  visible  on  your  map  display. 

3.  Using  the  FindMe  function,  display  your  symbol. 

4.  Determine  the  range  and  azimuth  FROM  YOU  to  BM  147. 

5.  Zoom  in  on  your  symbol  until  Mount  Paran  is  no  longer  visible  on  your  map  display. 

6.  Using  FindX.  find  Company  B.  When  you  find  the  unit  symbol  on  the  map,  click  on  it. 

7.  Using  FindX.  find  the  B  Team  Grenadier  for  Co.  C,  1®*  Pit,  1®‘  Sqd,  and  click  on  his  symbol. 

8.  You  are  the  commander.  Company  A.  Display  all  of  your  platoon  leaders. 

9.  Use  pan  to  locate  the  second  platoon  leader.  Click  on  his  symbol. 


Note  on  availability  of  individuals  and  units  in  the  map  area: 

•  Company  D  is  not  available. 

•  Only  four  complete  squads  are  available  for  display: 

>  Company  A,  3'^'^  Platoon,  Weapons  Squad 

>  Company  B,  2^^  Platoon,  2"'*  Squad 

>  Company  C,  1®*  Platoon,  1®‘  Squad 

>  Company  C,  2"'^  Platoon,  Weapons  Squad 


Instructions  for  the  Map  Final  Exam 

At  beginning  of  each  task,  there  is  a  red  banner  that  says: 


Click  NEXT  to  begin  the  exercise. 


Once  you  click  NEXT,  the  timer  for  the  exercise  starts.  You  must  click  NEXT  to  start  each  exercise.  You 
will  not  progress  unless  you  click  NEXT,  and  none  of  your  responses  will  be  scored. 

Tip.  Take  the  time  to  figure  out  the  answer  or  determine  your  approach  to  solving  the  question  before 
clicking  the  NEXT  button. 

In  the  Range  and  Azimuth  questions,  you  must  physically  move  the  cursor  to  each  answer  box  by 
clicking  on  it.  You  cannot  tab  to  the  other  box. 

Be  patient;  don’t  rush  when  answering. 
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Appendix  C 

Soldier  Background  Results 


Table  C-1 

Descriptive  Statistics  on  Age  of  each  Soldier  Group 


Soldier  Group 

N 

M 

Mdn 

SD 

Range 

ANCOC 

29 

31.59 

30 

3.40 

27-39 

BNCOC 

48 

27.75 

27 

3.32 

22-38 

lOBC 

42 

23.14 

23 

1.92 

21-29 

OSUT 

48 

19.58 

19 

2.36 

17-27 

.1  r*  1 

other  group. 


Table  C-2 

Number  of  Soldiers  by  Rank  in  Each  Soldier  Group 


Rank 

Soldier  Group 

OSUT 

BNCOC 

ANCOC 

lOBC 

Private 

All  48 

NA 

NA 

Sergeant 

15 

Staff  Sergeant 

33 

0 

Sergeant  First  Class 

28 

2d  Lieutenant 

All  42 

Note.  No  data  on  rank  from  1  soldier  in  ANCOC. 

Of  the  lOBC  students,  17  (41%)  received  their  commission  through  Reserve  Officer  Training 
Corps,  24  (57%)  from  the  US  Army  Military  Academy,  and  1  (2%)  from  the  US  Air  Force 
Academy. 


Table  C-3 

Descriptive  Statistics  on  Years  in  Active  Service 


Soldier  Group 

N 

M 

Mdn 

SD 

Range 

ANCOC 

29 

11.51 

11.9 

2.21 

9.00- 18.17 

BNCOC 

48 

7.47 

7.0 

2.60 

4.25-19.67 

lOBC 

42 

1.93 

0.8 

2.29 

0.00-7.17 

OSUT 

Not  asked  because  not  applicab 

le 

Note.  FortheOSU" 

r  soldiers,  3 1  (65%)  had 

a  high  school  education,  3  (6%)  had  technical 

school  training,  10  (21%)  had  less  than  4  years  of  college,  3  (6%)  had  a  college  degree,  and  1 
(2%)  hadaGED. 
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Table  C-4 

Percentage  of  Soldiers  Using  a  Computer  in  Different  Phases  of  Their  Formal  Education 


y©  Use  Computer 


Soldier  Group 

Grade 

School 

Junior 

High 

High 

School 

Technical 

School 

College 

Not  Use 

OSUT 

54% 

71% 

83% 

2% 

23% 

4% 

BNCOC 

6% 

15% 

58% 

2% 

40% 

19% 

ANCOC 

0% 

10% 

24% 

0% 

52% 

24% 

lOBC 

29% 

50% 

76% 

0% 

95% 

0% 

Table  C-5 

Number  of  Educational  Settings  Where  Soldiers  Used  a  Computer 


Soldier 

#  Educational  Settings  (%  soldiers) 

Group 

0 

1 

2 

3 

4-5 

OSUT 

21% 

27% 

33% 

15% 

BNCOC 

19% 

52% 

23% 

4% 

2% 

ANCOC 

24% 

69% 

3% 

3% 

0% 

lOBC 

0% 

24% 

29% 

21% 

26% 

Av  A  "V  T 

lyOie,  IViCall  IlUlllUCi  lui  cav/ii  wo-d.  x  — 

0.86;  lOBC  -  2.52.  F(3, 163)  =  26.90,  p  =  .0000.  All  courses  differed  from  each  other  except 
OSUTandlOBC. 


Table  C-6 

Percentage  of  Soldiers  Indicating  Computer  Ownership  and  Current  Use  of  a  Computer 


Soldier  Group 

%  Own  a 
Computer® 

%  Use 
Computer 
Now** 

Where  Currently  Use  Computer 

Home 

Work/Unit 

Tmg  Facility 

OSUT 

58% 

63% 

56% 

6% 

21% 

BNCOC 

67% 

98% 

69% 

63% 

42% 

ANCOC 

97% 

90% 

76% 

38% 

lOBC 

93% 

95% 

95% 

21% 

24% 

"  Own:  ff(3)  =  19.63,  p  =  .0002. 
Use:  x\3)  =  34.99,  p  =  .0000. 
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Table  C-7 

Descriptive  Statistics  on  the  Sum  of  Feature  Use  Ratings 


Soldier 

Sum  of  Feature  Use  Ratings 

Group 

N 

M 

Mdn 

SD 

Range 

Interquartile  Range 

OSUT 

48 

19.48 

21.5 

7.59 

0-28 

16-21.5 

BNCOC 

48 

17.50 

19.0 

7.53 

0-28 

14-23 

ANCOC 

29 

20.31 

21.0 

6.08 

0-28 

18-25 

lOBC 

42 

20.57 

21.5 

5.13 

2-28 

•  O  UL 

17-21.5 

_ i?9  ic 

Note.  The  7  features  were  rated  on  a  0  to  4-point  scale,  ranging  from  “never  used”  to  “daily 
use.”  Maximum  score  was  28  representing  daily  use  of  all  7  features;  minimum  score  was  0 


indicating  a  soldier  never  used  any  of  the  7  features. 
F(3, 163)=  1.84,/?  =  .1409. 


Table  C-8 

Percentage  of  Soldiers  Indicating  Different  Levels  of  Typing  Skill 


Self  Ratings  of  Typing  Skill 

Soldier 

Hunt  &  Peck 

Hunt  &  Peck 

Type  Slowly 

Type  Quickly 

Group 

Slowly 

Quickly 

OSUT 

6% 

48% 

25% 

21% 

BNCOC 

19% 

35% 

38% 

8% 

ANCOC 

17% 

33% 

30% 

20% 

lOBC 

0% 

36% 

19% 

45% 

Note.  x^(9)  =  27.83,  p  =  . 0010 


Table  C-9 

Descriptive  Statistics  on  Self-Ratings  of  Typing  Skill 


Soldier 

Self-Ratings  of  Typing  Skill 

Group 

N 

M 

Mdn 

SD 

Interquartile  Range 

OSUT 

’Is 

2.60 

2.00 

0.89 

2.0-3 .0 

BNCOC 

48 

2.35 

2.00 

0.89 

2.0-3.0 

ANCOC 

30 

2.53 

2.50 

1.01 

2.0-3.0 

lOBC 

42 

3.09 

3.00 

0.91 

,1 

2.0-4.0 

OSUT. 
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Table  C-10 

Percentage  of  Soldiers  Indicating  Different  Levels  of  Computer  Skill 


Self-Ratings  of  Computer  Skill 

Good  w 

Good  w 

1  Progm 

Several 

Bill 

Soldier 

N 

Novice 

1  softw 

several 

Lang  + 

Progm 

Gates 

Group 

program 

Soft  Progr 

Software 

Lang+Soft 

hire  me 

OSUT 

48 

38% 

25% 

35% 

0% 

2% 

0% 

BNCOC 

48 

44% 

25% 

29% 

2% 

0% 

0% 

ANCOC 

30 

40% 

20% 

40% 

0% 

0% 

0% 

lOBC 

42 

12% 

24% 

45% 

14% 

5% 

0% 

Table  C-11 

Descriptive  Statistics  on  Self-Ratings  of  Computer  Skill 


Self-Ratings  of  Computer  Skill 

Soldier 

N 

M 

Mdn 

SD 

Range 

Interquartile 

Group 

Range 

OSUT 

48 

2.04 

0.97 

1-5 

1-3 

BNCOC 

48 

1.89 

2.00 

0.91 

1-4 

1-3 

ANCOC 

30 

2.00 

2.00 

0.91 

1-3 

1-3 

lOBC 

42 

2.76 

3.00 

1.01 

1-5 

2-3 

Note.  Scale:  Novice  =  1,  One  software  program  =  2;  Several  software  program  =  3,  One 


program  language  +  software  =  4,  Program  languages  +  software  =  5;  Bill  Gates  hire  me  =  6. 
F(3, 164)  =  7.38,/?  =  .0001;  lOBC  mean  rating  higher  than  ANCOC,  BNCOC  and  OSUT. 
Chi-square  test  was  also  significant,  (12)  =  27.48,/?  =  .0065. 


Table  C-12 

Descriptive  Statistics  on  Icon  Test  Score 


Soldier 

Group 

Icon  Test  Scores 

M 

Mdn 

Range 

SD 

Interquartile 

Range 

Mean% 

Correct 

OSUT 

8.92 

8.50 

1-15 

3.39 

6.5-12 

BNCOC 

7.97 

7.75 

1-15 

3.48 

5.5-10.75 

44% 

ANCOC 

9.53 

9.50 

2-15 

3.12 

7.5-11 

53% 

lOBC 

11.17 

11.00 

4-15.5 

2.58 

10-11 

62% 

Note.  F(3, 164)  =  7.89,/?  =  .0001.  lOBC  mean  score  higher  than  ANCOC,  BNCOC,  and 


OSUT;  ANCOC  mean  score  higher  than  BNCOC. 
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Table  C-13 

Weapon  Symbols:  Percent  who  Answered  Correctly  and  Percent  who  Made  no  Response 


Weapon 

Symbol 

OSUT 

BNCOC 

ANCOC 

lOBC 

All  Groups 

%  Soldiers  Answering  Correctly 

M16Rifle/M4 

4% 

17% 

10% 

45% 

19% 

M240B/M60 

4% 

15% 

10% 

24% 

13% 

Anti-tank- 

Javelin 

2% 

17% 

20% 

24% 

15% 

M203 

0% 

2% 

3% 

31% 

9% 

Mortar 

0% 

13% 

33% 

21%  ^ 

15% 

%  Soldiers 

Not  Responding 

M16Rifle/M4 

96% 

44% 

10% 

26% 

48% 

M240B/M60 

96% 

54% 

30% 

36% 

57% 

Anti-tank 

Javelin 

98% 

69% 

73% 

62% 

76% 

M203 

98% 

71% 

70% 

33% 

69% 

Mortar 

100% 

58%  1 

57% 

38% 

65% 

Note.  For  each  soldier  group,  the  percentage  of  soldiers  answered  but  answered  incorrectly  is 
100%  minus  %  Correct  and  %  No  Response. 


Table  C-15 

Unit  Symbols:  Percent  who  Answered  Correctly  and  Percent  who  Made  no  Response 


Unit  Symbol 

OSUT 

BNCOC 

ANCOC 

lOBC 

All  Groups 

%  Soldiers  Answering  Correctly 

Squad 

2% 

27% 

79% 

65% 

39% 

Platoon 

0% 

13% 

67% 

55% 

29% 

Company 

2% 

4% 

73% 

43% 

26% 

%  Soldiers  not  Responding 

Squad 

88% 

42% 

7% 

12% 

41% 

Platoon 

85% 

38% 

0% 

12% 

38% 

Company 

83% 

44% 

0% 

24% 

42% 

•1  rvnn/ 

Note.  For  each  soldier  group,  the  percentage  who  answered,  but  answered  incorrectly  is  100% 
minus  the  percentage  correct  plus  the  percentage  of  no  responses. 


Table  C-16 

Unit  Symbols:  Frequency  Distribution 


#  Correct 

OSUT 

BNCOC 

ANCOC 

lOBC 

%  Soldiers  Within  Group 

0 

97%  n 

69% 

13% 

33% 

1 

0% 

21% 

10% 

12% 

2 

2% 

8%  n 

20% 

14% 

3 

0% 

2% 

57% 

41% 

Mean  (3  symbols) 

0.04 

0.44 

2.20 

1.62 

SD 

0.29 

0.74 

1.09 

1.32 

N 

48 

48 

30 

■1  •  1 

42 

1  _  .L  _  J 

Because  of  the  nonnormal  distribution  of  scores,  a  Chi-square  analysis  was  also  conducted. 
Scores  were  divided  at  0  and  greater  than  zero.  x^(3)  =  73.22,/?  =  .0001.  It  appeared  that  each 
group  differed  from  each  other  group,  based  on  the  difference  between  the  observed  and 


expected  frequencies. 


Table  C-17 

Map  Reading:  Percent  Soldiers  who  Answered  Correctly 


Item 

OSUT 

BNCOC 

ANCOC 

lOBC 

All  Groups 

8-digit  grid  #1 

33% 

96% 

93% 

78% 

73% 

8-digit  grid  #2 

29% 

94% 

97% 

71% 

Terrain  feature 

75% 

87% 

71% 

66% 

Hard-surface  road 

81% 

81%  1 

67% 

66% 

75% 

Distance  in  meters 

60% 

77% 

67% 

74% 

Grid  azimuth 

79% 

94% 

97% 

95% 

91% 
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Table  C-18 

Map  Reading:  Frequency  Distribution 


#  Correct 

OSUT 

BNCOC 

ANCOC 

lOBC 

%  Soldiers  Within  Group 

0 

8% 

0% 

0% 

0% 

1 

8% 

0% 

2 

17% 

0% 

2% 

3 

23% 

2% 

3% 

17% 

4 

17% 

19% 

19% 

5 

17% 

40% 

43% 

33% 

6 

10% 

40% 

33% 

29% 

Mean  (6  items) 

3.29 

5.17 

5.07 

4.69 

SD 

1.74 

0.81 

0.83 

1.14 

N 

48 

48 

30 

42 

_ THP 

F(3,164)  =  24.43,  p  =  .0000.  lOBC,  ANCOC,  and  BNCOC  means  each  greater  than  OSUT 


mean. 


Table  C-19 

Experience  with  the  Battle  Roster  System:  %  Soldiers 


OSUT 

BNCOC 

ANCOC 

lOBC 

All  Groups 

Trained  on  Battle 
Roster  System 

0% 

13% 

21% 

5% 

9% 

Used  Battle  Roster 
System 

0% 

31% 

38% 

7% 

18% 

C-1 


Table  C-20 

Summary  of  Item  Responses  for  Tendency  to  Solve  Problems  Independently 


%  Soldiers  Responding  to  Each  Alternative 

Item  or  Task 

Complete 
With  No 
Instructions 

Instructions 
Only  If 
Needed 

Read 

Instructions 

First 

Find 
Someone 
Else  To  Do  It 

Not 

Applicable 

Change  a  Tire  on  a  New 
Vehicle 

57% 

30% 

13% 

Find  Your  Seat  at  a 
Ballgame  or  Concert 

33% 

41% 

21% 

1% 

4% 

Change  Settings  on  a  New 
Digital  Watch 

27% 

48% 

25% 

Cook  a  Frozen  Dinner 

24% 

15% 

58% 

1% 

1% 

Put  Up  a  New  Tent 

19% 

51% 

30% 

— 

1% 

Play  a  New  Video  or 
Computer  Game 

18% 

48% 

27% 

1% 

7% 

Put  Together  a  New 
Bicycle 

15% 

45% 

40% 

1% 

Respond  to  a  Survey 

15%  n 

23% 

4% 

1% 

Program  a  New  V  CR 

11% 

38% 

1% 

1% 

Put  Together  a  Piece  of 
Furniture  that  Requires 
Assembly 

8% 

33% 

58% 

1% 

— 

Drive  Unfamiliar  Route  to 
a  New  Destination  More 
Than  100  Miles  From 
Home 

8% 

26% 

65% 

— 

1% 

Set-up  a  New  Computer 

7% 

41% 

39% 

10% 

4% 

Take  a  New,  Over-The- 
Coimter  Medication 

4% 

4% 

83% 

1% 

8% 

Learn  a  New  Computer 
Software  Application 

4% 

31% 

55% 

4% 

6% 

Fill  Out  Tax  Forms 

3% 

11% 

49% 

33% 

4% 

rr^-f 

Note.  N=\6Z.  Items  ordered  by  percent  responding  to  “complete  with  no  instructions.”  The 
two  items/tasks  shaded  in  gray  were  eliminated  from  additional  analyses  (take  a  new  over-the- 
counter  medication,  and  fill  out  tax  forms).  Row  percentages  sum  to  100%,  within  rounding 


error. 
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Table  C-21 

Mean  Number  of  Tasks  Checked  in  each  Response  Option  to  the  Working  Independently  Items 


OSUT 

BNCOC 

ANCOC 

lOBC 

All 

Soldiers 

Response  Option 

M 

(SD) 

(Range) 

M 

(SD) 

(Range) 

M 

(SD) 

(Range) 

M 

(SD) 

(Range) 

M 

(SD) 

(Range) 

Tasks  usually 

3.10 

2.35 

1.87 

2.24 

2.45 

completed  without 

(3.32) 

(2.65) 

(2.27) 

(2.55) 

(2.79) 

instructions^ 

(0-10) 

(0-10) 

(0-9) 

(0-10) 

(0-13) 

Tasks  usually 

4.71 

4.46 

4.63 

4.98 

4.69 

attempted  wo/ 

(2.45) 

(2.74) 

(3.55) 

(2.68) 

(2.79) 

instructions  until 
problem  occurs'’ 

(0-10) 

(0-13) 

(0-13) 

(0-12) 

(0-13) 

Tasks  usually 

4.96 

5.50 

5.93 

5.24 

5.36 

performed  only  after 

(3.06) 

(3.02) 

(3.30) 

(3.03) 

(3.08) 

reading  instructions  ( 
or  given  guidance)' 

(0-12) 

(0-12) 

(0-13) 

(0-11) 

(0-13) 

Tasks  usually  given  to 

0.17 

0.29 

0.20 

0.24 

0.23 

someone  else  to  do'* 

(0.48) 

(0-2) 

(0.65) 

(0-3) 

(0.48) 

(0-2) 

(0.53) 

(0-2) 

(0.54) 

(0-3) 

Tasks  where  soldier 

0.06 

0.40 

0.37 

0.31 

0.27 

could  not  respond  (e.g.. 

(0.32) 

(0.89) 

(0.96) 

(0.64) 

(0.73) 

not  done  before)  — 

Not  applicable' 

(0-2) 

(0-5) 

(0-4) 

(0-3) 

,  •  1  •  _ 

(0-5) 

X'  xl-  ... 

Note.  Because  soldiers  were  required  to  select  one  of  the  response  options  listed  for  each  of  the 


13  tasks,  means  in  each  column  sum  to  13. 

“Completed  without  instructions:  F(3,  160)  =  1.55, p  =  .204. 

'’Attempted  without  instructions  until  problem  occurs:  F(3, 160)  =  .26,/?  =  .853. 
'Performed  only  after  reading  instructions:  F(3,  160)  =  .80,/?=.50. 

Task  usually  given  to  someone  else:  F(3,  160)  =  .45,  p  -  .72. 

'Not  applicable:  F(3,  160)  =  2.10,/?  =  .10. 
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Table  C-22 

Analysis  of  Variance  on  Tendency  to  Work  Independently:  Task  Approach  Strategies  by  Soldier 
Group  and  Map  Training  Conditions 


Factor 

df 

MS 

F 

P 

Complete  With  No  Instructions 

Map  Conditions 

2, 156 

0.352 

0.044 

.957 

Soldier  Group 

3, 156 

11.83 

1.48 

.222 

Interaction 

6, 156 

3.07 

0.384 

.888 

Instructions  Only  If  Needed 

Map  Conditions 

2, 156 

0.951 

0.116 

0.890 

Soldier  Group 

3, 156 

3.64 

0.444 

0.722 

Interaction 

6, 156 

2.36 

0.287 

0.942 

Read  Instructions  First 

Map  Conditions 

2, 156 

2.54 

0.264 

.768 

Soldier  Group 

3, 156 

10.66 

1.11 

.347 

Interaction 

6, 156 

9.73 

1.01 

.419 

Find  Someone  Else  To  Do  It 

Map  Conditions 

2, 156 

0.151 

0.500 

.607 

Soldier  Group 

3, 156 

0.078 

0.259 

.855 

Interaction 

6, 156 

0.263 

0.870 

.519 

Not  Applicable 

Map  Conditions 

2, 156 

0.311 

0.570 

.567 

Soldier  Group 

3, 156 

0.970 

1.77 

.154 

Interaction 

6, 156 

0.066 

0.120 

.994 

Table  C-23 

Mean  Ratings  on  Facility  with  Computer  Items 


Item 

OSUT 

BNCOC 

ANCOC 

lOBC 

All 

Soldiers 

M 

(SD) 

(Range) 

M 

(SD) 

(Range) 

M 

(SD) 

(Range) 

M 

(SD) 

(Range) 

M 

(SD) 

(Range) 

Have  a  “knack”  or 
“feel”  for  finding  my 
way  around  a 
computer 

5.58 

(2.11) 

(1-10) 

5.06 

(2.86) 

(1-10) 

5.23 

(2.36) 

(1-10) 

6.19 

(2.33) 

(2-10) 

5.52 

(2.46) 

(1-10) 

Even  if  know  one  way 
to  perform  a  task, 
usually  figure  out  a 
shortcut 

5.48 

(2.56) 

(1-10) 

4.54 

(2.88) 

(1-10) 

A.ll 

(2.79) 

(1-10) 

5.71 

(2.78) 

(1-10) 

5.14 

(2.77) 

(1-10) 

1  c  rr\ 

Shortcut;  F(3, 164)  =  1.80,/?  =  .1499.  Ratings  on  a  10-point  scale,  1  =  does  not  describe  me  at 
all  to  10  =  describes  me  completely. 
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Appendix  D 

Examples  of  Lessons,  Exercises,  and  Exams 


Disclaimer. 

The  border  on  the  displays  contains  the  words  “Land  Warrior.” 

The  symbols  and  map  interface  DO  NOT  correspond  to  the  current  version 

of  the  Land  Warrior  system. 

The  experiment  was  based  on  a  prototype  version  of  the  Land  Warrior  map. 
For  purposes  of  the  experiment, 

we  generated  our  own  detailed  version  of  how  the  map  might  function 
and  the  symbols  used  to  identify  individuals  and  xmits. 


D-1 


D-2 


D-3 


Battle  Roster  Numbering  System 


■V\IAYS  is  a  5  Letter/Number 
Combination: 


|i/ ' 


Battle  Roster  Numbering  System  | 

\ 

\ 

ISC  r 

osifion 

5th  Position  A 

BATTALION 

INDIVIDUAL  y 

2d  Position 

4th  Position 

\ 

CO?;^PANY 

SQUAD 

\ 

3d  Position  Each  soldier  has  !' 

PLATOON  a  position  number  1; 

code. 

I; 

Y 

% 

At  what  unit  level  do  these  two  battle  roster  numbers  differ? 


Is 


AA1434 

AA1424 


t40TB.:  This  is  s  tivmzi 


w&l 


Click  on  the  correct  answer 


D-4 


D-5 


D-6 


Bill 

JfMM 

I;  ^||||}  #ife;:sia8wfewJ|lif^|vB^ 

-  .  1  ,  _  » Sc^i^  L^der  | 

:r>Br^  •^AI^F^Tean  ^ 

sif 

I  f'  /B  ^  -  ThJ'^pBTJcev'of the  mcfividuate 


1  Alpha  Team  Leader 

2  Alpha  Teanj  SAW  Goriner  Alpha 

3  Alpha  Tejurs  Grenadier  Team 

4  Alpha  Team  Rifleman 

5  Bravo  Tc^i  Leader 

6  Bravo  Team  $AWGuii.ner  g^yQ 

7  Bravo  Team  Grejiacfief  Team 

8  Srtivo  Tc^i  Rifleman 


vife 

t;i-A 


-"'L  I 


1^3 


mSm 

feeamil 


Bravo  I 

,  'Team  I 


I  r^£|" ; 


fcii 


'1 


n  Rifle  Squad  Battle  Roster  C^es 

-|i  ^  To  complete  the  graphic  symbol  foreach  squad  member:  \  . 
4v:l’  ?  Take  the  individuals  battie'rostenn^bfc^,,  >£'3^  '''  '  ■ 

%T<  , :’;,  vrrB'^T  '  /"B  '/ 

...and  add  the  symbolfor  the  weapon  earned  Mvfdui..; 


''  I 


D-IO 


D-11 


Examples  of  additional  lessons  and  final  exam  questions 
are  not  provided.  The  exam  questions  followed  the 
formats  illustrated.  However,  there  was  no  repetition  of 
specific  items  given  in  the  training  exercises. 


MAP  LESSONS, 
EXERCISES,  AND  EXAM. 
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Review 

The  Find  X  ftjnction  allows  you  to  find  a  unit  or  an  individual  within  a  unit. 

To  actuate  the  Find  X  function  to  find  a  particular  unit  or  person,  click  the 
down  arrow  next  to  the  Find  X  button. 

Continue  your  choices  until  you  find  the  indh/idual  or  unit  you  seek,  then 
click  on  it. 

The  corresponding  symbol  and  battle  roster  graphic  will  then  appear 
centered  on  your  map. 


D-18 


D-19 


D-20 


D-22 


D-23 


D-24 


A'  L«ndWMCKK  '  ttaining  Emetiwgti* 


B  ME  X 

r«oot  rnrfWo  r»v!x 


Message 

}  -  "  If 

Mailbox 

u''^\  ^  m  — 

'vO-^ 

FL0e286$39  J 

:Zd 

jiiQ^22S!Si3i 

Higo 


The  range  aiimuih  mode  wiil  help  you  determine  a  direcdoit  and  distance  to 
;  a  selected  point  from  a  selected  point 


ill^ 


HI  I 


vs 


.mmmwMRmR 


A  LandWmtot  ■  Ttammo  Expenwent 


^  ‘Q  0  ME  X 

1,/  In  pflr,  T;*tw  r«Hx_ . . P'!rr,.. . ' P.‘^*r:i2i 


'fvf 


iflM 

WMSM 


"I  Yiiu  me  the  1st  PUlnon  tender  of  A  Company  and  you  want  lor^ow  how  far 
:  >  and  in  what  direction  your  first  squad  leader  is  located. 


fm 


Range  and  Azimuth  Mode 
Review 

1 .  Click  on  the  “Range  Azimulh"  button.  The  range  azimirth  capability  is 
now  functional  and  the  button  changes  to  "Cancel". 

2.  Click  on  a  start  point  and  then  an  end  point.  The  range  and  azimuth  will 
appeeir  in  meters  emd  degrees.  At  this  point,  the  button  also 
automatically  changes  back  to  “Range  Azimuth". 

3.  Individuals:  measure  ftom/to  the  base  stem 

4.  Units:  measure  fromAo  the  lower  left  comer  of  the  symbol. 

5.  Can  only  measure  one  distance  at  a  time. 

6.  To  determine  another  range  azinrjuth,  click  on  the  "Range  Azimulh" 
button  again. 

7.  Whenever  you  click  on  the  "Cancel"  button,  the  function  is  disabled. 


“j  j  aicir  MEXT  ofier  completion  of  reading  the  review  points.  There  isno  exercise 

■  1  for  lesson  M.  The  final  exam,  however,  vdll  have  situations  where 
range/'arimuth  will  be  used. 
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D-26 


D-28 


D-29 


Appendix  E 

Results  on  Code  and  Map  Training 
Codes:  Exercise  and  Final  Exam  Scores 


Table  E-1  ^  , 

ANOVA  on  Code  Exercises  and  Code  Final  Exam  Scores:  High-Low  Demand  Conditions  by 

Soldier  Group 


Factor 

df 

MS 

F 

P 

Weapon  and  Unit  Symbols 

High-Low  Demand 

1,160 

43.08 

1.59 

.2091 

Soldier  Group 

3,160 

1.97 

0.07 

.9745 

Interaction 

3, 160 

3.10 

0.11 

.9516 

BR  Numbering  System 

High-Low  Demand 

1, 159 

75.47 

0.76 

3m/ 

Soldier  Group 

3,159 

121.05 

1.22 

.3049 

Interaction 

3,159 

179.82 

1.81 

.1475 

Company  through  Squad  BR 

High-Low  Demand 

1,159 

141.85 

2.46 

.1186 

Soldier  Group 

3,159 

75.18 

1.30 

.2748 

Interaction 

3,159 

4.17 

0.07 

.9746 

Rifle  and  Weapons  Squads  Codes 

High-Low  Demand 

1,159 

3,843.57 

23.53 

.0000 

Soldier  Group 

3, 159 

1,023.15 

6.26 

.0004 

Interaction 

3,159 

177.31 

1.09 

.3570 

Key  Leader  and  Unit  Codes 

High-Low  Demand 

1, 159 

0.39 

0.004 

.9482 

Soldier  Group 

3, 159 

501.69 

5.51 

.0012 

Interaction 

3,159 

12.58 

0.14 

.9371 

Final  Exam  on  Codes 

High-Low  Demand 

1, 159 

186.86 

1.48 

.2240 

Soldier  Group 

3,159 

814.49 

6.49 

.0003 

Interaction 

3, 159 

23.28 

0.18 

.9061 
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Table  E-2 

Descriptive  Stotistics  for  High  and  Low  Demand  Conditions  on  the  Code  Exercises  and  Code 
Final  Exam  Scores  (%  Correct) 


Condition 

M 

SD 

Mdn 

Min-Max 

N 

Weapon  and  Unit  Symbols 

Low  Demand 

97.70 

4.87 

100 

63-100 

85 

High  Demand 

96.69 

5.36 

100 

71-100 

83 

BR  Numbering  System 

Low  Demand 

94.64 

11.24 

100 

20-100 

84 

High  Demand 

95.78 

8.71 

100 

60-100 

83 

Company  through  Squad  BR 

Low  Demand 

97.65 

3.38 

100 

87-100 

84 

High  Demand 

95.74 

10.15 

100 

30-100 

83 

Rifle  and  Weapons  Squads  Codes‘^  _  _ _ _ 

Low  Demand  93.09  7.07  94  60-100  84 

High  Demand  82.89  17.60  _ 88 _ 31-100 _ 83 

Key  Leader  and  Unit  Codes  _ _ _ 

Low  Demand  89.73  9.09  92  54-100  84 

High  Demand  |  89.71  |  10.58  |  92  |  38-100  |  83 

Final  Exam  on  Codes _ _ _ _ _ _ _ _ 

Low  Demand  87.19  12.00  92  48-100  84 

High  Demand  89.30 _ 11-31 _ 92 _ 40-100 _ 83 

Rifle/Weapons  Squads.  F(l,  159)  =  23.53, p=  .0000;  Low  scores  greater  than  High. 
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Table  E-3 

Descriptive  Statistics  for  Soldier  Group  on  the  Code  Exercises  and  Code  Final  Exam  Scores 
(%  Correct) 


Soldier  Group 

M 

SD 

Mdn 

Min-Max 

N 

Weapon  and  Unit  Symbols 

lOBC 

97.32 

3.99 

100 

83-100 

42 

ANCOC 

97.08 

4.66 

100 

79-100 

30 

BNCOC 

97.39 

6.35 

100 

63-100 

48 

OSUT 

96.96 

5.07 

100 

71-100 

48 

BR  Numbering  System 

lOBC 

97.56 

5.38 

100 

80-100 

41 

ANCOC 

93.00 

16.43 

100 

20-100 

30 

BNCOC 

95.00 

8.25 

100 

70-100 

48 

OSUT 

94.79 

9.45 

100 

60-100 

48 

Company  through  Squad  BR 

lOBC 

98.69 

2.22 

100 

93-100 

41 

ANCOC 

96.44 

6.37 

97 

67-100 

30 

BNCOC 

95.97 

11.11 

100 

30-100 

48 

OSUT 

95.90 

6.78 

98 

63-100 

48 

Rifle  and  Weapons  Squads  Codes  “ 

lOBC 

92.33 

12.09 

97 

31-100 

41 

ANCOC 

91.52 

10.44 

94 

57-100 

30 

BNCOC 

88.51 

11.65 

91 

40-100 

48 

OSUT 

81.67 

18.13 

88 

34-100 

48 

Key  Leader  and  Unit  Codes  ^ 

lOBC 

92.77 

8.54 

96 

71-100 

41 

ANCOC 

91.53 

7.70 

92 

71-100 

30 

BNCOC 

90.54 

9.44 

92 

67-100 

48 

OSUT 

85.16 

12.18 

88 

38-100 

48 

Final  Exam  on  Codes  ‘ 

lOBC 

92.68 

9.33 

96 

48-100 

41 

ANCOC 

91.20 

10.10 

96 

60-100 

30 

BNCOC 

87.92 

12.25 

92 

44-100 

48 

OSUT 

82.92 

11.95 

84 

56-100 

48 

^  Rifle/Weapons  Squads.  F(3, 159)  =  626.  p  =  .0004;  ANCOC,  BNCOC,  and  lOBC  greater 
than  OSUT. 


Key  Leaders/Unit.  F(3, 159)  =  5.51,/?  =  .0012;  ANCOC,  BNCOC,  and  lOBC  greater  than 
OSUT. 

Final  Exam.  Flf,  159)  =  6.49,/?  =  .0003;  ANCOC,  BNCOC,  and  lOBC  greater  than  OSUT; 
lOBC  greater  than  BNCOC. 
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Codes:  Exercise  and  Exam  Times 


Table  E-4 

ANOVA  on  Time  to  Complete  Code  Instruction  and  Code  Final  Exam:  High-Low  Demand 
Conditions  by  Soldier  Group  (all  soldiers) 


Factor 

df 

MS 

F 

P 

Weapon  and  Unit  Symbols 

High-Low  Demand 

1, 160 

0.89 

0.43 

.5140 

Soldier  Group 

3, 160 

36.46 

17.41 

.0000 

Interaction 

3, 160 

2.19 

1.05 

.3725 

BR  Numbering  System 

High-Low  Demand 

1, 159 

1.06 

0.43 

.5123 

Soldier  Group 

3, 159 

3.47 

1.41 

.2420 

Interaction 

3, 159 

3.76 

1.53 

.2098 

Company  through  Squad  BR 

High-Low  Demand 

1, 159 

0.19 

0.04 

.8431 

Soldier  Group 

3, 159 

47.74 

10.02 

.0000 

Interaction 

3, 159 

3.14 

0.66 

.5787 

Rifle  and  Weapons  Squads  Codes 

High-Low  Demand 

1,  159 

764.15 

16.32 

.0001 

Soldier  Group 

3,  159 

606.17 

12.95 

.0000 

Interaction 

3,  159 

117.64 

2.51 

.0605 

Key  Leader  and  Unit  Codes 

High-Low  Demand 

L  159 

0.37 

0.01 

.9055 

Soldier  Group 

3,  159 

235.27 

8.87 

.0000 

Interaction 

3,  159 

4.25 

0.16 

.9228 

Final  Exam  on  Codes 

High-Low  Demand 

1,  159 

9.59 

0.99 

.3200 

Soldier  Group 

3,  159 

19.03 

1.97 

.1198 

Interaction 

3, 159 

15.69 

1.63 

.1849 
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Table  E-5 

Descriptive  Statistics  for  High  and  Low  Demand  Conditions  on  Time  to  Complete  Code 
Instruction  and  Code  Final  Exam  (in  minutes  and  fraction  of  a  minute,  for  all  soldiers) 


1  Condition 

M 

SD 

Mdn 

Min-Max 

N 

1  Weapon  and  Unit  Symbols 

Low  Demand 

6.02 

1.49 

5.90 

2.02-10.38 

85 

High  Demand 

6.15 

1.80 

5.82 

2.72-12.58 

83 

BR  Numbering  System 

Low  Demand 

4.96 

1.86 

2.22-16.42 

High  Demand 

4.78 

1.24 

4.85 

1.57-7.25 

83  1 

Company  through  Squad  BR 

Low  Demand 

9.39 

1.83 

9.27 

5.97-13.70 

84 

High  Demand 

9.27 

2.78 

8.88 

3.50-20.18 

83 

Rifle  and  Weapons  Squads  Codes'"  \ 

Low  Demand 

17.37 

4.30 

16.92 

10.28-28.47 

84 

High  Demand 

21.94 

9.93 

19.68 

5.82-67.40 

83 

1  Key  Leader  and  Unit  Codes 

Low  Demand 

16.55 

4.76 

16.29 

8.57-29.27 

84 

High  Demand 

16.38 

6.10 

14.95 

5.13-48.03 

83 

1  Final  Exam  on  Codes 

Low  Demand 

10.03 

3.00 

9.11 

5.57-23.72 

84 

High  Demand 

9.41 

3.30 

8.58 

4.60-26.17 

83 

Note.  Includes  time  for  soldiers  who  repeated  exercises.  No  repetition  of  the  final  exam  was 


allowed. 

^  Rifle/Weapons  Squads.  F(l,  159)  =  16.32,p  =  0001;  Low  faster  than  High. 


Table  E-6 

Descriptive  Statistics  for  Soldier  Groups  on  Time  to  Complete  Code  Instruction  and  Code  Final 
Exam  (in  minutes  and  fraction  of  a  minute,  for  all  soldiers) 


Soldier  Group _  M  |  SD 

Weapon  and  Unit  Symbols  “ 
lOBC  n  5^03  I  L28 

ANCOC  5.68  1.69 

BNCOC  6.18  1.19 

OSUT _  7.18  I  1.63 

BR  Numbering  System 
lOBC  ^  4A5  I  228 

ANCOC  5.10  1.46 

BNCOC  4.92  1.17 

OSUT _  5.06  I  1.22 

Company  through  Squad  BR 
lOBC  1  289  ^  136 

ANCOC  9.03  1.98 

BNCOC  9.71  2.57 

OSUT _  10.36  I  2.28 

Rifle  and  Weapons  Squads  Codes  ^ 

lOBC  r  16.01  ^  3A4 

ANCOC  17.17  6.07 

BNCOC  19.47  6.05 

OSUT  _  24.46  I  10.72 

Key  Leader  and  Unit  Codes 
lOBC  ^  13.74  ^  433 

ANCOC  15.79  5.14 

BNCOC  16.36  3.72 

OSUT _  19.32  I  6.45 

Final  Exam  on  Codes 

lOBC  ^  936  ^  434 

ANCOC  8.60  1.58 

BNCOC  9.97  3.05 

OSUT _  10.30  I  3.04 

®  Weapon/Unit  Symbols.  F(3,  160)  =  17.41,/?  = 
faster  than  OSUT.  lOBC  faster  than  BNCOC. 
*’  Company-Squad.  F(3,  159),=  10.02,/?=  .0000 
lOBC  faster  than  ANCOC  and  BNCOC. 
Rifle/Weapons  Squads.  2’(3,  159),=  12.95,/?  = 
than  OSUT;  lOBC  faster  than  BNCOC. 

Key  Leader/Unit.  F(3,  159),  =  8.87,/?  =  .0000; 
OSUT;  lOBC  faster  than  BNCOC. 


Mdn  Min-Max 

4.87  I  2.02-  8.59 

5.27  3.78-12.58 

5.94  4.28-11.77 

7.31  I  2.52-11.18 

4.05  I  2.22-16.42 

4.81  2.93-9.37 

4.98  2.33-7.32 

4.90  1.56-8.57 


733  I  5.97-13.70  41 

8.92  6.10-13.63  30 

9.25  6.17-20.18  48 

10.16  I  3.50-17.37  |  48 

15.35  I  10.67-27.32  ^  4l 

15.82  10.28-37.60  30 

18.53  11.07-42.77  48 

22.33  I  5.82-67.40  |  48 

12.38  I  8.07-25.23  ^  41 

16.21  8.57-24.68  30 

16.54  10.17-28.22  48 

18.30  I  5.13-48.03  |  48 

08  ^  5.52-26.17  I  4l 

8.15  6.23-12.80  30 

9.44  5.56-23.72  48 

9.83  I  4.60-17.53  |  48 

=  .0000;  lOBC,  ANCOC,  and  BNCOC  each 

)0;  lOBC  and  ANCOC  faster  than  OSUT. 

=  .0000;  lOBC,  ANCOC,  and  BNCOC  faster 

3;  lOBC,  ANCOC,  and  BNCOC  faster  than 
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Table  E-7 

ANOVA  on  Time  to  Complete  Code  Instruction  and  Code  Final  Exam:  High-Low  Demand 
Conditions  by  Soldier  Group  for  Soldiers  who  did  not  Repeat  an  Exercise 


Factor 

df 

MS 

F 

P 

Weapon  and  Unit  Symbols 

High-Low  Demand 

1, 156 

0.09 

0.06 

.8084 

Soldier  Group 

3,156 

34.95 

23.47 

.0000 

Interaction 

3, 156 

1.06 

0.71 

.5449 

BR  Numbering  System 

High-Low  Demand 

1, 152 

0.71 

0.31 

.5791 

Soldier  Group 

3,152 

2.31 

1.00 

.3929 

Interaction 

3, 152 

4.03 

1.75 

.1599 

Company  through  Squad  BR 

High-Low  Demand 

1, 154 

6.19 

1.60 

.2075 

Soldier  Group 

3, 154 

37.79 

9.77 

.0000 

Interaction 

3,154 

1.94 

0.50 

.6816 

Rifle  and  Weapons  Squads  Codes 

High-Low  Demand 

1, 140 

214.18 

8.71 

.0037 

Soldier  Group 

3, 140 

212.38 

8.64 

.0000 

Interaction 

3, 140 

25.83 

1.05 

.'hill 

Key  Leader  and  Unit  Codes 

High-Low  Demand 

1,122 

0.03 

0.00 

.9627 

Soldier  Group 

3,122 

126.41 

7.91 

.0000 

Interaction 

3, 122 

21.17 

1.32 

.2694 

Final  Exam  on  Codes 

High-Low  Demand 

1, 106 

8.99 

0.84 

.3601 

Soldier  Group 

3, 106 

7.83 

0.73 

.5331 

Interaction 

3, 106 

13.14 

1.23 

.3001 

Note.  Final  exam  times  included  only  those  soldiers  who  did  not  repeat  any  exercises. 
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Table  E-8 

Descriptive  Statistics  for  High  and  Low  Demand  Conditions  on  Time  to  Complete  Code 
Instruction  and  Code  Final  Exam  for  Soldiers  who  did  not  Repeat  an  Exercise  (in  minutes  and 
fraction  of  a  minute) 


1  Condition 

M 

SD 

Mdn 

N 

1  Weapon  and  Unit  Symbols  \ 

Low  Demand 

5.98 

1.46 

5.89 

84 

High  Demand 

5.94 

1.44 

5.76 

80 

Battle  Roster  Numbering:  System  \ 

Low  Demand 

4.85 

1.80 

4.68 

81 

High  Demand 

4.72 

1.18 

4.68 

79 

1  Company  through  Squad  BR 

Low  Demand 

9.29 

1.83 

9.27 

84 

High  Demand 

8.89 

2.39 

8.68 

78 

Rifle  and  Weapons  Squads  Codes'" 

Low  Demand 

16.98 

16.66 

78 

High  Demand 

19.34 

6.51 

18.61 

70 

1  Key  Leader  and  Unit  Codes  \ 

Low  Demand 

15.06 

4.25 

14.34 

58 

High  Demand 

15.09 

4.34 

14.59 

72 

1  Final  Exam  on  Codes 

Low  Demand 

9.71 

2.90 

8.61 

54 

High  Demand 

9.26 

3.55 

8.18 

60 

Note.  Final  exam  times 

included  only  those  soldiers  who  d 

id  not  repeat  any  exercises. 

®  Rifle/Weapons  Squads.  F(l,  140)  =  8.71,  p  =  0037;  Low  condition  faster  than  High. 
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Table  E-9 

Descriptive  Statistics  for  Soldier  Groups  on  Time  to  Complete  Code  Instructions  and  Final  Exam 
for  Soldiers  who  did  not  Repeat  an  Exercise  (in  minutes  and  fraction  of  a  minute) 


Soldier  Group 

M 

SD 

Mdn 

N 

Weapon  and  Unit  Symbols  " 

lOBC 

5.03 

1.29 

4.88 

42 

ANCOC 

5.44 

1.09 

5.23 

29 

BNCOC 

5.99 

0.76 

5.92 

46 

OSUT 

7.09 

1.54 

7.22 

47 

Battle  Roster  Numbering  System 

lOBC 

4.45 

2.28 

4.05 

41 

ANCOC 

4.91 

1.25 

4.76 

28 

BNCOC 

4.78 

1.06 

4.85 

45 

OSUT 

4.99 

1.20 

4.83 

46 

Company  through  Squad  BR  ^ 

lOBC 

7.90 

1.56 

7.73 

41 

ANCOC 

8.88 

1.81 

8.87 

29 

BNCOC 

9.44 

2.26 

9.24 

46 

OSUT 

10.13 

2.07 

10.07 

46 

Rifle  and  Weapons  Squads  Codes '' 

lOBC 

16.03 

3.48 

15.37 

40 

ANCOC 

16.66 

4.85 

15.58 

28 

BNCOC 

18.38 

4.33 

17.93 

43 

OSUT 

21.43 

7.14 

20.97 

37 

Key  Leader  and  Unit  Codes 

lOBC 

12.83 

3.66 

12.35 

37 

ANCOC 

14.52 

5.09 

12.43 

23 

BNCOC 

15.39 

2.88 

15.13 

39 

OSUT 

17.57 

4.64 

16.63 

31 

Final  Exam  on  Codes 

lOBC 

9.62 

4.28 

8.28 

36 

ANCOC 

8.52 

1.62 

8.15 

20 

BNCOC 

9.73 

2.47 

9.35 

33 

OSUT 

9.69 

3.45 

8.63 

25 

Note.  Final  exam  times  included  only  those  soldiers  who  did  not  repeat  any  exercises. 


®  Weapon  &  Unit  symbols.  F{3, 156)  =  23.47.  p  =  .0000;  lOBC,  ANCOC,  and  BNCOC  each 
faster  than  OSUT,  and  lOBC  faster  than  BNCOC. 

*’  Company-Squad  BR.  F(3, 154),  =  9.77,/?  =  .0000;  lOBC  faster  than  OSUT  and  BNCOC. 

^  Rifle/Weapons  Squads.  F(3, 140),  =  8.64,/?  =  .0000;  lOBC,  ANCOC,  and  BNCOC  faster 
than  OSUT. 

^  Key  Leader  and  Unit.  F(3,  122),  =  7.91,/?  =  .0000;  lOBC  and  ANCOC  faster  than  OSUT; 
lOBC  faster  than  BNCOC. 


E-9 


Map:  Exercise  and  Final  Exam  Scores 


Table  E-10 

ANOVA  on  Map  Exercises  and  Map  Final  Exam  Scores:  High-Low-Exploratory  Conditions  by 
Soldier  Group 


Factor 

df 

MS 

F 

P 

Zoom,  Pan,  Find  Me 

High-Low  Demand 

1,74 

305.29 

2.66 

.1071 

Soldier  Group 

3,74 

227.95 

1.99 

.1233 

Interaction 

3,74 

33.26 

0.29 

.8326 

FindX 

High-Low  Demand 

1,71 

362.46 

1.32 

.2540 

Soldier  Group 

3,71 

782.61 

2.86 

.0431 

Interaction 

3,71 

224.42 

0.82 

.4878 

Display 

High-Low  Demand 

1,70 

2992.16 

4.07 

.0474 

Soldier  Group 

3,70 

1846.66 

2.51 

.0655 

Interaction 

3,70 

573.97 

0.78 

.5086 

Final  Exam  on  Map 

High-Low- 

2, 142 

1704.01 

6.84 

.0014 

Exploratory 

Conditions 

Soldier  Group 

3, 142 

819.17 

3.29 

.0226 

Interaction 

6.  142 

92.18 

0.37 

.8971 
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Table  E-11 

Descriptive  Statistics  for  High  and  Low  Demand  Conditions  on  the  Map  Exercises,  and  for  High, 
Low  and  Exploratory  Conditions  on  the  Map  Final  Exam  (%  correct) 


Condition 

M 

SD 

Mdn 

Min-Max 

N 

Zoom,  Pan,  Find  Me 

Low  Demand 

94.96 

6.59 

100 

75-100 

43 

High  Demand 

91.45 

13.98 

100 

33-100 

39 

Find  X 

Low  Demand 

87.70 

15.96 

92 

33-100 

43 

High  Demand 

84.91 

18.20 

92 

8-100 

37 

Display  “ 

Low  Demand 

84.23 

20.30 

87 

13-100 

42 

High  Demand 

71.18 

34.44 

87 

0-100 

36 

Final  Exam  on  Map  ^ 

Low  Demand 

78.27 

13.02 

82 

40-100 

40 

High  Demand 

75.27 

15.22 

80 

23-71 

34 

Exploratory 

66.62 

17.70 

73 

17-80 

80 

“  Display.  F(l,  70)  =  4.07, .0474;  Low  scores  greater  than  High  scores. 

**  Final  Exam.  F(2,  142)  =  6.84,p  =  .0014;  Low  scores  greater  than  both  High  and  Exploratory 


scores,  High  scores  greater  than  Exploratory  scores. 
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Table  E-12 

Descriptive  Statistics  for  Soldier  Group  on  the  Map  Exercises  and  Final  Exam  (%  correct) 


Soldier  Group _ 

Zoom,  Pan,  Find  Me 

TOBC 
ANCOC 
BNCOC 
OSUT 
FindX  “ 
lOBC 
ANCOC 
BNCOC 

OSUT _ 

Display 

lOBC 

ANCOC 

BNCOC 

OSUT _ 

Final  Exam  on  Map  ^ 

lOBC 

ANCOC 

BNCOC 

OSUT 

"FindX.  F(3,71)  =  2.86,p 
Final  Exam.  F(3,  142)  =  3. 


97.22 

88.89 

93.84 

92.03 

■8ra“ 

81.94 

92.75 

80.07 

90.00 

84.37 
69.57 

73.37 

78.70 

72.00 

68.11 

68.11 


6.63 

17.72 

6.75 

10.80 

24.32 

11.32 
20.11 

~Wn 

19.31 

33.25 

30.22 

15.04 

10.67 

17.98 

18.30 


Min-Max 

75-100 

33-100 

83-100 

67-100 

67-100 

8-100 

58-100 

13-100 

13-100 

38-100 

0-100 

0-100 

40-100 

33-90 

17-90 

23-97 


=  .043 1 ;  BNCOC  scores  greater  than  ANCOC  and  OSUT. 
19,  p  =  .0226;  lOBC  scores  greater  than  BNCOC  and  OSUT. 
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Map:  Times  on  Lessons-Exercises  and  Final  Exam 


Table  E-13 

ANOVA  on  Time  to  Complete  Map  Training:  High-Low-Exploratory  Conditions  by  Soldier 
Group 


Factor 

df 

MS 

F 

P 

Zoom,  Pan,  Find  Me 

High-Low  Demand 

1,74 

29.61 

5.34 

.0235 

Soldier  Group 

3,1  A 

31.72 

5.72 

.0014 

Interaction 

3,  74 

3.14 

0.57 

.6385 

Find  X 

High-Low  Demand 

1,71 

5.85 

0.91 

.3432 

Soldier  Group 

3,71 

28.96 

4.50 

.0059 

Interaction 

3,71 

1.78 

0.28 

.8415 

Display 

High-Low  Demand 

1,70 

11.54 

0.94 

.3355 

Soldier  Group 

3,  70 

113.85 

9.28 

.0000 

Interaction 

3,70 

2.06 

0.17 

.9177 

Range  and  Azimuth  (lesson  only,  no  exercises) 

High-Low  Demand 

1,75 

29.87 

20.14 

.0000 

Soldier  Group 

3,75 

1.22 

0.82 

.4862 

Interaction 

3,75 

1.05 

0.71 

.5500 

Final  Exam  on  Map 

High-Low-Exploratory 

2, 138 

395.46 

46.34 

.0000 

Soldier  Group 

3, 138 

16.86 

1.97 

.1205 

Interaction 

6.  138 

3.47 

0.41 

.8738 

Map  Training 

(All  lessons  and  exercises  for  Low  and  High  Demand;  Working  with  map  for  Exploratory) 

High-Low  Exploratory 

2,  143 

31,051.71 

622.65 

.0000 

Soldier  Group 

3, 143 

476.51 

5.55 

.0000 

Interaction 

6, 143 

63.06 

1.26 

.2774 

Total  Time  in  Map  Phase 

High-Low  Exploratory 

2,  138 

22,359.18 

296.45 

.0000 

Soldier  Group 

3, 138 

577.50 

7.66 

.0000 

Interaction 

6, 138 

43.07 

0.57 

.7528 
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Table  E-14 

Descriptive  Statistics  on  Time  on  Map  Lesson/Exercises  and  Map  Final  Exam  (times  in  minutes 
and  fractions  of  minute) 


Condition _ M 

Zoom,  Pan,  Find  Me  “ _ 

Low  Demand  15.17 

High  Demand  13.92 

FindX _ _ 

Low  Demand  14.03 

High  Demand _ 13.37 

Display _ _ 

Low  Demand  22.29 

High  Demand  21.66 

Range  Azimuth  ^ 

Low  Demand  2.98 

High  Demand  4.23 

Final  Exam  on  Map  _ 

Low  Demand  1 5 .07 

High  Demand  15.33 

Exploratory  22.02 

Map  Training 

Low  Demand  54.54 

High  Demand  53.65 

Exploratory _ 12.63 

Total  Time  ^ ^ _ 

Low  Demand  69.28 

High  Demand  70.81 

Exploratory _ 34.65 


9.21 

10.26 

8.57 


14.78 

13.77 

TUT 

13.43 

~2i2n 

21.64 


15.40 
15.16 
20.75 

55.30 

54.40 
11.08 

69.40 
70.65 
33.58 


Min-Max 

11.03-26.00 

8.40-20.53 

9.08-22.98 

7.88- 20.52 

14.88- 29.42 
12.75-29.72 

0.82-6.62 

1.63-9.23 

11.28-19.35 

10.00-20.72 

12.23-42.38 

38.20-74.82 

33.02-72.62 

2.10-34.88 

50.27-83.90 

45.90-96.53 

19.65-59.00 


Note.  Range-Azimuth  times  are  based  only  on  the  lesson  as  there  were  no  practice  exercises. 
^  Zoom  and  Pan.  F(l,  74)  =  5.3A,p=  .0235;  Low  faster  than  High. 

Range  and  Azimuth.  F(L  75)  =  20.14,  p  =  .0000;  Low  faster  than  High. 

Final  Exam.  F{2,  138)  =  46.34, /?  =  .0000;  Low  and  High  faster  than  Exploratory. 

Map  training.  F{2, 143)  =  622.65,/?  =  .0000;  Exploratory  faster  than  High  and  Low. 

®  Total  time.  F(2,  138)  =  296.45,  p  =  .0000;  Exploratory  faster  than  High  and  Low 
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Table  E-15 

Descriptive  Statistics  on  Time  in  Map  Lessons/Exercises  and  Map  Final  Exam  by  Soldier  Group 


(times  in  minutes  and fraction  of  minutes) 


Condition 

M 

SD 

Mdn 

Min-Max 

N 

Zoom,  Pan,  Find  Me  " 

lOBC 

12.84 

2.06 

12.72 

8.40-17.28 

21 

ANCOC 

14.64 

2.01 

12.82 

9.75-16.48 

15 

BNCOC 

15.25 

2.02 

14.72 

11.83-19.18 

23 

OSUT 

15.47 

3.03 

14.92 

11.37-26.00 

23 

FindX^  ~ 

lOBC 

12.29 

1.94 

Yini 

7.88-15.10 

21 

ANCOC 

13.16 

2.01 

12.82 

9.75-16.48 

12 

BNCOC 

14.02 

2.97 

13.75 

9.08-22.98 

23 

OSUT 

15.02 

2.67 

14.57 

10.77-20.52 

23 

Display  ' 

lOBC 

19.00 

3.05 

19.53 

12.75-23.80 

20 

ANCOC 

20.63 

4.22 

21.42 

12.95-26.23 

12 

BNCOC 

23.23 

3.06 

23.92 

16.00-29.42 

23 

OSUT 

24.09 

3.68 

24.65 

15.38-29.72 

23 

1  Range  and  Azimuth 

lOBC 

3.43 

1.41 

3.03 

1.97-6.62 

21 

ANCOC 

3.26 

1.25 

3.52 

0.82-5.57 

15 

BNCOC 

3.68 

1.52 

3.38 

1.38-9.23 

24 

OSUT 

3.84 

1.19 

3.75 

1.52-5.58 

23 

Final  Exam  on  Map 

lOBC 

17.50 

4.58 

16.67 

10.00-32.13 

41 

ANCOC 

18.71 

6.32 

17.62 

11.65-42.38 

25 

BNCOC 

20.23 

5.75 

19.41 

11.77-37.57 

44 

OSUT 

18.54 

5.06 

17.65 

10.37-32.28 

41 

Map  Training 

lOBC 

29.69 

19.51 

26.69 

2.1-59.33 

40 

ANCOC 

30.88 

21.95 

22.68 

4.82-68.48 

25 

BNCOC 

34.06 

22.75 

28.13 

6.53-72.62 

45 

OSUT 

35.94 

23.73 

34.88 

3.1-74.82 

45 

Total  Time  ' 

lOBC 

47.60 

17.56 

46.21 

20.30-74.25 

40 

ANCOC 

50.24 

20.71 

50.27 

20.95-93.12 

25 

BNCOC 

54.73 

20.49 

49.87 

26.00-96.53 

44 

OSUT 

52.14 

20.86 

55.15 

19.65-82.50 

41 

^  Zoom  and  Pan.  F(3,  74)  =  5.72,  p  =  .0014;  lOBC  faster  than  ANCOC,  BNCOC,  and  OSUT. 
Find  X.  F(3, 71)  =  4.50,;?  =  .0059;  lOBC  faster  than  BNCOC  and  OSUT;  ANCOC  faster 


than  OSUT. 

'Display.  F(3, 70)  =  9.28,;?=  .0000;  ANCOC  and  lOBC  faster  than  OSUT;  lOBC  faster  than 
BNCOC. 

^  Map  Training.  7^(3, 143)  =  5.55,/?  =  .0000,  lOBC  faster  than  BNCOC  and  OSUT. 

'  Total  Time.  F(3, 138)  =  7.66,/?  =  .0000;  lOBC  faster  than  BNCOC  and  OSUT. 
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Correlations 


Table  E-16 

Correlations  Among  Code  Scores  (Low  and  High  Demand) 


Code  Scores 

Sym 

BR 

Co/Sqd 

Rifle/Wpn 

Ldr/Umt 

Exam 

Sym 

1.00 

.07 

-.01 

.00 

-.06 

.08 

BR 

1.00 

.13 

.21** 

.15* 

.17* 

Co/Sqd 

1.00 

.24** 

.23** 

.24** 

Rifle/Wpn 

1.00 

43*** 

Ldr/Unit 

1.00 

43*** 

Exam 

1.00 

Note.  A^=  167. 

*;7<.05.  **;?<. 01.  ***p<.001. 


Table  E-17 

Correlations  Among  Code  Times  (Low  and  High  Demand) 


Code  Times 

Sym 

BR 

Co/Sqd 

Rifle/Wpn 

Ldr/Unit 

Exam 

Total 

Sym 

BR 

Co/Sqd 

Rifle/Wpn 

Ldr/Unit 

Exam 

Total 

1.00 

50*** 

1.00 

59*** 

.50*** 

1.00 

4g*** 

40*** 

55*** 

1.00 

53*** 

42*** 

60*** 

.62*** 

1.00 

.19* 

.15* 

.23** 

.20** 

27*** 

1.00 

63  *  *  * 

50*** 

74*** 

.86*** 

82*** 

44*** 

1.00 

_ _ 

Note.  167.  Times  represent  total  of  lesson  and  exercise  times,  as  separate  times  on  code 


lessons  and  exercises  were  not  available. 

*p<.Q5.  **p<.Q\.  ***p<.001. 

Table  E-18 

Correlations  Between  Code  Scores  and  Code  Times 


Code 

Scores 

Code  Times 

Sym 

BR 

Co/Sqd 

Rifle/Wpn 

Ldr/Unit 

Exam 

Total 

Sym 

BR 

Co/Sqd 

Rifle/Wpn 

Ldr/Unit 

Exam 

-  32*** 

-.03 

-.05 

-.19* 

-.17* 

_  29*** 

-.11 

-.33*** 

-.09 

-.15 

-.10 

-.18* 

-.15* 

-.17* 

-.46*** 

-  27*** 
-.23** 

_  37*** 

-.05 

-.12 

-.16* 

-  34*** 
-.56*** 

_  37*** 

-.11 
-.14 
-.12 
_  49*** 

-.17* 

-  34*** 

.04 

-.08 

.03 

-.09 

-.05 

-.15* 

-.10 

-.19* 

-.18* 

-  42*** 

“  39*** 

-  42*** 

Note.  N=\6‘ 

7.  Correlai 

dons  between  time  and  scores  on  same  i 

topic  are  in  1 

bold  (diagonal) 

*p<.05.  **/><. 01.  ***/?<. 001. 
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Table  E-19 

Correlations  Among  Map  Scores  (Low  and  High  Demand) 


Map  Scores 

Zoom/Pan 

FindX 

Display 

Exam 

Zoom/Pan 

1.00 

33** 

FindX 

1.00 

33** 

31** 

Display 

*** 

Exam 

1.00 

Note.  N  =  71-82.  (some  missing  data  on  some  variables). 
*p<.05.  **p<.0l.  ***;?<. 001. 


Table  E-20 

Correlations  Among  Map  Times  (Low  and  High  Demand) 


Map  Lesson  Times 

Zoom/Pan 

FindX 

Display 

Range/Azm 

Exam 

Zoom/Pan 

1.00 

gj*** 

.58*** 

.22* 

g  1  *** 

FindX 

1.00 

y  j  *** 

42*** 

72*** 

Display 

1.00 

23*** 

73*** 

Range/ Azm 

1.00 

.55*** 

Exam 

1.00 

Map  Exercise  Times 

Zoom/Pan 

1.00 

29*** 

27*** 

NA 

47*** 

FindX 

1.00 

.54*** 

NA 

56*** 

Display 

1.00 

NA 

66*** 

Range/ Azm 

NA 

NA 

Exam 

1.00 

Note.  =  71-82  (some  missing  data  on  some  variables).  There  were  no  range/azimuth 
exercises.  Correlations  between  lesson  and  exercise  times  for  the  map  functions  were  as  follows: 


Zoom/Pan/Find  Me  =  .30*;  Find  Others  =  .31*;  Display  =  .18 
*/7<.05.  **j!7<.01.  ***;?<. 001. 


Table  E-21 

Correlations  Between  Map  Scores  and  Map  Times  (Low  and  High  Demand) 


Map 

Map  Times 

Scores 

Map  Lesson 

Map  Exercise/Exam 

Zoom/Pan 

.15  w/  zoom/pan  lesson  time 

-  35***  Yv/  zoom/pan  exercise  time 

FindX  1 

-.04 

^  4g*** 

Display 

.12 

„  43*** 

Exam 

-.  14  w/  total  time  in  map  training 

.06  w/  time  to  take  exam 

Note.  N  =  71-82  (some  missing  data  on  some  variables). 
*j!7<.05.  **;?<. 01.  ***p><.001. 
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Table  E-22 

Correlations  Between  Map  Scores  and  Map  Times  (Exploratory  condition  only) 


Time  Exploring 

Time  taking  Exam 

Final  Exam  Score 

.26* 

..  1  ♦  li* 

-.13 

_ _ 

Note.  N  =  .79.  Correlation  between  time  exploring  and  time  taking  exam  was  .07. 
*  p<.05 


Table  E-23 

Correlations  Among  Background  Variables 


Computer  Survey 

Military  Knowledge 

Wo 

rk  Inde 

penden 

Hy 

Own 

Freq 

Self 

Rate 

Icon 

Wpns 

Unit 

Sym 

Map 

Read 

BR 

Use 

BR 

Tm 

BR 

Knack 

Short 

K&S 

Work 

Ind 

Own 

1.00 

.66 

.42 

.33 

.23 

.29 

.31 

.12 

.16 

.16 

.19 

.33 

.36 

.37 

.00 

Freq 

1.00 

.45 

.49 

.12 

.20 

.19 

-.09 

.06 

.06 

.06 

.55 

.52 

.57 

-.05 

Rate 

1.00 

.52 

.30 

.27 

.33 

.00 

-.04 

.03 

-.01 

.57 

.55 

.60 

-.12 

Icon 

1.00 

.22 

.32 

.32 

.03 

-.02 

-.04 

-.03 

.53 

.51 

.55 

-.05 

Wpns 

1.00 

.42 

.86 

.13 

-.13 

-.09 

-.13 

.09 

.08 

.09 

.09 

Unit 

1.00 

.83 

.22 

.07 

.16 

.12 

.20 

.16 

.19 

-.03 

Sym 

1.00 

.21 

.04 

.04 

.00 

.17 

.14 

.17 

.03 

Map 

1.00 

.18 

.11 

.17 

-.06 

-.08 

-.08 

.12 

BR 

Use 

1.00 

.49 

.81 

.03 

.09 

.06 

-.05 

BR 

Tm 

1.00 

.91 

.07 

.09 

.09 

i 

.04 

BR 

1.00 

,05 

.10 

.09 

-.07 

Knack 

1.00 

.76 

.93 

“•17 

Short 

1.00 

.94 

-118 

K&S 

1.00 

-.19 

Work 

Ind 

1  .  •  1 

- - 

1.00 

iV  -  - - - ^ - - C? 

were  replaced  by  the  variable  immediately  following  them  in  the  table. 

Correlations  greater  than  .16  were  significant  at  .05  levels;  those  greater  than  .31  were  significant 
at  the  .01  level. 


Table  E-24 

Correlations  Between  Tendency  to  Work  Independently  and  Map  Measures  (Exploratory 
condition  only) 


Map  Final  Exam 
Score 

Time  Taking  Map 
Final  Exam 

Exploratory  Time 

Tendency  to  Work 
Independently 

.04 

.15 

-.12 

Note.  N  =  79. 
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Table  E-25 

Correlations  between  Background  Measures  and  Final  Exam  Scores 


Criterion  Measures  and  Training  Conditions 

Background  Measures 

Code  Exam 

Map  Exam 

Low 
{n  =  84) 

High 
{n  =  83) 

Low 
(n  =  40) 

High 
(«  =  34) 

Exp 
(n  =  80) 

Computer  background 

.21 

.06 

.14 

.18 

-Own  computer 

.10 

-Use  computer  features 

.11 

.17 

.25 

.20 

.23* 

-Self-rating  of  expertise 

.18 

.28** 

.29* 

.32* 

.28** 

-Icon  score 

.26* 

.29** 

.35* 

.29* 

-Short  cut/Knack 

.17 

.13 

.38** 

.03 

.24* 

BR 

.17 

.01 

-.10 

-.12 

-.04 

Military  knowledge 

.28 

33** 

-Symbols 

.28** 

.21* 

.23 

-Map  Reading 

.24* 

.29** 

.01 

.36* 

.20 

Tendency  to  work 

-.05 

-.07 

independently 

-.01 

.01 

-.09 

Multiple  R  -  all  predictors 

.42 

.46 

.51 

.52 

.52 

F 

1.77 

1.20 

.95 

2.83 

df 

9,  73 

9,  30 

9, 23 

9, 69 

P 

.088 

.333 

.504 

.007 

Multiple  R  -  stepwise 

.33 

.41 

.38 

.46 

.47 

F 

4.99 

7.49 

6.52 

4.08 

10.94 

df 

1,  80 

1,49 

1,38 

2,30 

2, 76 

P 

.009 

.001 

.015 

.027 

.000 

*  p<.05.  **  p<.01.  ***p<.001. 
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